-- Logs begin at Wed 2022-08-10 22:00:06 UTC, end at Wed 2022-08-10 23:39:44 UTC. -- Aug 10 22:51:16.031453 dv-cinder-851449-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 22:51:16.041928 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 22:51:16 2022] *** Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: nodename: dv-cinder-851449-5-iscsi Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: machine: x86_64 Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: clock source: unix Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: pcre jit disabled Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: detected number of CPU cores: 4 Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: current working directory: / Aug 10 22:51:16.043972 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: detected binary path: /usr/bin/uwsgi-core Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: your processes number limit is 47680 Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: your memory page size is 4096 bytes Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: detected max file descriptor number: 2048 Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: lock engine: pthread robust mutexes Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: thunder lock: enabled Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 22:51:16.044531 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 22:51:16.083197 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: Python main interpreter initialized at 0x5608750f39c0 Aug 10 22:51:16.083197 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: python threads support enabled Aug 10 22:51:16.083197 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: your server socket listen backlog is limited to 100 connections Aug 10 22:51:16.083197 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: your mercy for graceful operations on workers is 90 seconds Aug 10 22:51:16.083536 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 22:51:16.083621 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: *** Operational MODE: preforking *** Aug 10 22:51:16.083889 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: *** uWSGI is running in multiple interpreter mode *** Aug 10 22:51:16.083889 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: spawned uWSGI master process (pid: 81627) Aug 10 22:51:16.084088 dv-cinder-851449-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 22:51:16.085172 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: spawned uWSGI worker 1 (pid: 81638, cores: 1) Aug 10 22:51:16.085172 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: spawned uWSGI worker 2 (pid: 81639, cores: 1) Aug 10 22:51:16.085172 dv-cinder-851449-5-iscsi devstack@keystone.service[81627]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 22:51:21.657147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:51:21.657147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: from cryptography.utils import int_from_bytes Aug 10 22:51:21.658101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:51:21.658101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: from cryptography.utils import int_from_bytes Aug 10 22:51:21.658208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:51:21.658208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: from cryptography.utils import int_from_bytes Aug 10 22:51:21.658208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:51:21.658208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: from cryptography.utils import int_from_bytes Aug 10 22:51:21.713763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 22:51:21.713935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 22:51:21.714007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 22:51:21.714084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 22:51:21.714151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] command line args: [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 22:51:21.714212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] command line args: [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 22:51:21.714459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 22:51:21.714547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 22:51:21.714547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ================================================================================ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 22:51:21.714700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ================================================================================ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 22:51:21.714772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] admin_token = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.714837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] admin_token = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_dir = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config_dir = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_source = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config_source = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] debug = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] debug = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.715717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.716009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.716109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.716178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.716243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.716302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_config_append = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_config_append = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.717960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_dir = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_dir = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.718690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.720979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721774 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.721901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] publish_errors = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.722948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] publish_errors = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] standard_threads = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] standard_threads = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_journal = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_json = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.723988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.724053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_stderr = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.724143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.724257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_syslog = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.724396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.724656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.724838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.724939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.725141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.725380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.725603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_journal = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.725697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.725855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.726183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.726291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_json = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.726366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_stderr = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.726513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.726604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_syslog = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.726822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:51:21.726921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.727070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.727184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.727392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.727567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.727737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.727836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.728995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.729059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.729253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.729395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.729507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.729747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.729984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.730203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.730288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.730502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.730591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.730783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.730894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.731995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.732951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.733946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.734952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.735112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.735367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.735602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.735709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.735833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.736065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.736307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.736511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.736733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.736935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.737139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.737387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.737591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.737794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.738013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.738232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.738342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.738671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.738883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.739112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.739334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.739556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.739723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.739921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.740157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.740374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.740542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.740739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.740904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.741942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.742080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.742242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.742384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.742526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.742721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.742871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.743892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.744872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.745910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.746006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.746102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.746284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.746505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.746616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.748795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.748943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.749922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.750945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.751923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.752053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.752218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.752369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.752506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.752637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.752766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.753424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.753596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.753733 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.753893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.754938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.755921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.756985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.757149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.757286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.757442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.757598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.757732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.757885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.external = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.758951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.password = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.token = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.759974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.760935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.external = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.761901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.762243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.762389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.762557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.762557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.password = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.762838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.762969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.token = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.763906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.764860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.765983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.766140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.766273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.766403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.766543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.766731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.766910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.767997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.768985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.769793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.770865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.771953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.772904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.773040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.773040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.773297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.773297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.773555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.774912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.775938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.776868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.777942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.778984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.779969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.780109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.780253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.780506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.780668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.780781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.780881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.781936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.782917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.783015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.784648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.784829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.785023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.785226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.785437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.785595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.785745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.785884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.786926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.787188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.787324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.787471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.787610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.787747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.787884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.788868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.789984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.790121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.790258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.790404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.790542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.790697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.790889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.791866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.792999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.793136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.793273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.793488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.793628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.793767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.793904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.794041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.794178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.794391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.794760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.794913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.795052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.795268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.795482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.795643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.795785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.795924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.796061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.796198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.796381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.796702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.796854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.797952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.798089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.798225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.798398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.798811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.799003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.799144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.799281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.799429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.799649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.800505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.800636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.800728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.800810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.800887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.800963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.801777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.802980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.803908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.804918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.805964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.806798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.811855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.811992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.812932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.813983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.814968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.815042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.815117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.815191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.815274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.815879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.815957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.816969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:51:21.817562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 22:51:21.831850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.832084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.832312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.833377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.833607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.833826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.842316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.844061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.872878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.873107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.873205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.873334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.930787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.931234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.983780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.983959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.984223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:21.984458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.055948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.056032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.056164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.056257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.056374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.079988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.080092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.080203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.080294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.080414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.080496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.080626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.098575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.098958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.111982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.112065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.112722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.112815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.112935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.113018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.113147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.113238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:51:22.130184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.130391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:51:22.130628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.130771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.130860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:51:22.130958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:51:22.131187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.131274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.131385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:51:22.131475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.131655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.131743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.131915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.132000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.132175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.132429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.143370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.144165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.144251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.144438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.144523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.150573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.150696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.150696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.153162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.153280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.156410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.156700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.156996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.157286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.157580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.157804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.157900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.158119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.158205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.158409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.158732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.159170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.159951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.160228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.160500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.160769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.161046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.161276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.161480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.161565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.161879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.162165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.185538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.185774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.187083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.187310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.187398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.187595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.192026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.192200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.192316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.192488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.192603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.192769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.199263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.199363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.199554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.199641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.203596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.203702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.203890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.203977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.204178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.204277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.204462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.204547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.204739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.204825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.205085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.205179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.205257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.205332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.205524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.205610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.222276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.222386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.222546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.222636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.226452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.231266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.231357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.231437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.231527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.231605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.234394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.234493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.237529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.237626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.240666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.240768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.240964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.241926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.242099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.242195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.258591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.258721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.258903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.258989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.259169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.259254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.259430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.259516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.266372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.266494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.267946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.268038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.268219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.268305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.271574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.271680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.271857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.271943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.272150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.272235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.272440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.272526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.272716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.272802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.284707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.284821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.284993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.285930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.286112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.286195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.286381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.286468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.303180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.303307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.306100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.306226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.309141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.309255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.310737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.310848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.312225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.312324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.312517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.312636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.312802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.312905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.319825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.319936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.321322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.321413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.324270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.324373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.324555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.324642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.324836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.324921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.328635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.333243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.333340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.333580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.333722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.333872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.334008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.341014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.341116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:51:22.341297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.341386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.341583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.341668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.341856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.341950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.342970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.343975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.344152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.344237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.344432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.344516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.344688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.344773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:51:22.378844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5608750f39c0 pid: 81639 (default app) Aug 10 22:51:22.378979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5608750f39c0 pid: 81638 (default app) Aug 10 22:51:22.382147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a78868-26ff-488a-a4a2-f14a642f284c None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:22.382428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a78868-26ff-488a-a4a2-f14a642f284c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:22.382720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a78868-26ff-488a-a4a2-f14a642f284c None None] PATH_INFO: `/v3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:22.386943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1/1] 10.222.0.78 () {50 vars in 791 bytes} [Wed Aug 10 22:51:22 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:31.304404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:31.305119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:31.305119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:31.309408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1/2] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:51:31.316219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:31.316685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:31.317123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:31.392072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: INFO dbcounter [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] Registered counter for database keystone Aug 10 22:51:31.414696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG oslo_db.sqlalchemy.engines [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81639) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 22:51:31.420976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writer thread running {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 22:51:31.603778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG passlib.handlers.bcrypt [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81639) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 22:51:31.604120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG passlib.handlers.bcrypt [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81639) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 22:51:31.660731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:31.662385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:31.809613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 2/3] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 22:51:32.135000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:32.204998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: INFO dbcounter [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] Registered counter for database keystone Aug 10 22:51:32.224872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG oslo_db.sqlalchemy.engines [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81638) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 22:51:32.231126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writer thread running {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 22:51:32.350528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d092-16d5-499e-916a-9a97ccc3ed05 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:32.355233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:32.355897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:32.356181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:32.356597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:32.366087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:32.417849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING keystone.common.rbac_enforcer.enforcer [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 22:51:32.418131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a7766-ed8f-47fd-b967-49a906f7c7b0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:32.429171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 2/4] 10.222.0.78 () {58 vars in 1324 bytes} [Wed Aug 10 22:51:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:51:32.435325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:32.450283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de99b7-b608-4c5e-8a59-a79ccf3be8e0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:32.454903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:32.455992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:32.456269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:32.456539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] PATH_INFO: `/v3/projects/669ea2e08a3a4675928c19c0f5a75735` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:32.458095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] RBAC: Authorizing `identity:get_project(project_id=669ea2e08a3a4675928c19c0f5a75735)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:32.508822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 22:51:32.509158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a8a0f-109f-4dde-8c34-a6bb954a6907 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:32.513707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 3/5] 10.222.0.78 () {56 vars in 1207 bytes} [Wed Aug 10 22:51:32 2022] GET /identity/v3/projects/669ea2e08a3a4675928c19c0f5a75735 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:34.492771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b37f-ba4a-4e9e-b33a-43f685cd0dc5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:34.493262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b37f-ba4a-4e9e-b33a-43f685cd0dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:34.493346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b37f-ba4a-4e9e-b33a-43f685cd0dc5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:34.497403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 3/6] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:34.502982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:34.503389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:34.503534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:34.524446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:34.525972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:34.534211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 4/7] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 22:51:34.556827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b37f-ba4a-4e9e-b33a-43f685cd0dc5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:34.569402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b37f-ba4a-4e9e-b33a-43f685cd0dc5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:34.573508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1102e-5aba-4d69-abff-03c08c073665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:34.574095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1102e-5aba-4d69-abff-03c08c073665 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:34.574372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1102e-5aba-4d69-abff-03c08c073665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:34.574643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1102e-5aba-4d69-abff-03c08c073665 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:34.583009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1102e-5aba-4d69-abff-03c08c073665 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:34.584766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1102e-5aba-4d69-abff-03c08c073665 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:34.593429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 4/8] 10.222.0.78 () {58 vars in 1324 bytes} [Wed Aug 10 22:51:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:51:34.598716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:34.607666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3edb65-e49d-4c54-8b6c-b75742216991 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:34.612501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a25850-1eb6-401f-b73d-8e2964c4388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:34.613111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a25850-1eb6-401f-b73d-8e2964c4388c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:34.613392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a25850-1eb6-401f-b73d-8e2964c4388c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:34.613666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a25850-1eb6-401f-b73d-8e2964c4388c admin admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:34.615032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a25850-1eb6-401f-b73d-8e2964c4388c admin admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:34.616841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a25850-1eb6-401f-b73d-8e2964c4388c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:34.618347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 5/9] 10.222.0.78 () {56 vars in 1198 bytes} [Wed Aug 10 22:51:34 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:36.665340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04058edc-2888-44b4-9001-4ba38a218cf5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:36.665848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04058edc-2888-44b4-9001-4ba38a218cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:36.665922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04058edc-2888-44b4-9001-4ba38a218cf5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:36.667233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 5/10] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:36.672238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:36.672625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:36.672788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:36.691009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:36.692117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:36.708744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745979ed-ca8c-4fb4-ab48-1f83bfaec4ef None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:36.709027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745979ed-ca8c-4fb4-ab48-1f83bfaec4ef None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:36.709309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745979ed-ca8c-4fb4-ab48-1f83bfaec4ef None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:36.710620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 6/11] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:51:36.715439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:36.715732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:36.716010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:36.759590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG passlib.handlers.bcrypt [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81638) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 22:51:36.760016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG passlib.handlers.bcrypt [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81638) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 22:51:36.771260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 6/12] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:51:36.795902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:36.810987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:36.812607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:36.819739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:36.899962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da930cfd-c013-49d5-b082-0298a1bda1ef None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:36.911290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 7/13] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:51:36.932810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:36.933157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:36.933790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:36.934069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:36.934340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:36.945744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:36.953996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:36.955104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:36.956586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 7/14] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:51:36.961347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:37.036081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db57c5-e727-4e07-ac9b-11697c2cb6f2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.038695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:37.042220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:37.059890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806913e7-03e6-4c7e-b174-ee4bef3b31a2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.072079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.072936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.073219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.073493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.093107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.093703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.093981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.094260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.094966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:37.095265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.097197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.108713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 8/15] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:37.114189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:37.139630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.141468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.216089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb2b90-a6b4-47d7-92c1-366d4e2fc257 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.240902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 8/16] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:51:37.246867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:37.250108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:37.356377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba47771-f076-4aa3-972f-b333dbd6360d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.359088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.359713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.359989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.360257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.361403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.362994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.364587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 9/17] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:37 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:37.369979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:37.385334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:37.390626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.391510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.391793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.392066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.410489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.411985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:51:37.430125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:37.431158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:51:37.432024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:51:37.432024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:51:37.432024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:51:37.432024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 9/18] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:51:37.437001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:37.475115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703f0dc-6248-47b0-bdae-79bffd31f467 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.493576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:37.508938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.518531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.518531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.518531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.518531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.518531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.518531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 10/19] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:37.519875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:37.537310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9634fbc-c9f6-47ed-a339-27ad1a5ccbdd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.571441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.572044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.575758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.575758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.575758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:37.575758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.575758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fde6c7-e921-4d65-b390-e4ac6c4ee0bf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.592171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 10/20] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:37.599062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:37.623795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e711-c035-452c-8bcf-adf366ed4acb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:37.664009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:37.664662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:37.664944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:37.665218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:37.665960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.domain.id=default, user.id=e793eb6ffc5c41a282b74540f29c0a31) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:37.666270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:37.667808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92524bb-137b-4e72-86e6-7379deb02e77 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:37.675619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 11/21] 10.222.0.78 () {56 vars in 1322 bytes} [Wed Aug 10 22:51:37 2022] GET /identity/v3/role_assignments?user.id=e793eb6ffc5c41a282b74540f29c0a31&scope.domain.id=default&role.id=be98a6e301f144d88c992f65bca561fb => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:39.869643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b39c1-5421-430a-8874-704086c8bf6e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:39.870855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b39c1-5421-430a-8874-704086c8bf6e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:39.871337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b39c1-5421-430a-8874-704086c8bf6e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:39.874130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 11/22] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:39.882106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:39.883024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:39.883496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:39.905806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:39.907353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:39.991806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 12/23] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:51:40.027064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b39c1-5421-430a-8874-704086c8bf6e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:40.148527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b39c1-5421-430a-8874-704086c8bf6e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:40.176324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:40.192028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:40.200308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.200308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.200308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.207706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:40.209184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c2b19bb8-00fd-4dc3-a70d-9f26e97d9946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:51:40.225994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:40.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:51:40.227764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:51:40.227764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:51:40.227764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:51:40.227764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 12/24] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:40.233512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:40.249566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce70b-30a9-4af8-9f86-a3eb218bdfc8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.249913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce70b-30a9-4af8-9f86-a3eb218bdfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.250189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce70b-30a9-4af8-9f86-a3eb218bdfc8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.251678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 13/25] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:40.257687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.257963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.258230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.289420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:40.290966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:40.369430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:40.412519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd7c2-ebcf-4c7e-a651-6906313d3a61 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:40.426327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 14/26] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:51:40.471144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:40.501484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:40.503278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.503692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.504089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.505139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:40.505583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:40.508074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:40.549056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 13/27] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:40.567813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:40.815525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:40.818077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a372b3-a746-4e15-87e7-41ccd16ba7b4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:40.824757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df375b5-9771-429e-8fee-a6797466ecfb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:40.862252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:40.863125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.863544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.863952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.864848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:40.866452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:40.871912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:40.881137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cc07e-e689-4561-bcdf-a503a7500a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:40.882693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cc07e-e689-4561-bcdf-a503a7500a29 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.892055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cc07e-e689-4561-bcdf-a503a7500a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.892055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cc07e-e689-4561-bcdf-a503a7500a29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.897325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9cc07e-e689-4561-bcdf-a503a7500a29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ee9cc07e-e689-4561-bcdf-a503a7500a29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:40.904366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:40.905430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:40.906370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:51:40.908390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 15/28] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:51:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:40.909667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c24b4b-b0e1-4bdc-90a1-b01d8241b02f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.910066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c24b4b-b0e1-4bdc-90a1-b01d8241b02f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.910457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c24b4b-b0e1-4bdc-90a1-b01d8241b02f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.912696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 16/29] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:40.916997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43044-eecd-400d-b71a-ed1589b5c95d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:40.916997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43044-eecd-400d-b71a-ed1589b5c95d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:40.916997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43044-eecd-400d-b71a-ed1589b5c95d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:40.917722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 17/30] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:40.918299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43044-eecd-400d-b71a-ed1589b5c95d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:41.029988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43044-eecd-400d-b71a-ed1589b5c95d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:41.065717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:41.066253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.066534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.066853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.067511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:41.067865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:41.069304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c34755-6955-47e2-9524-c4aac80837fa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:41.086515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 18/31] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:41.086515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87406b4f-3000-44f4-baf0-d2f42c0fca14 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.086515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87406b4f-3000-44f4-baf0-d2f42c0fca14 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.086515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87406b4f-3000-44f4-baf0-d2f42c0fca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.102612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-87406b4f-3000-44f4-baf0-d2f42c0fca14 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:41.103816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87406b4f-3000-44f4-baf0-d2f42c0fca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:41.118626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:41.179752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 19/32] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:41.181411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.181917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.182392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.202589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:41.204197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:41.281739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 14/33] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:51:40 2022] POST /identity/v3/projects => generated 301 bytes in 714 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:41.290898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:41.293921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 20/34] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:41.296180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:41.470908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:41.490843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a96a9-44f3-489c-a019-af16372cac30 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:41.497618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0ff7-fd30-437f-84a0-90e50b30eba4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:41.531810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:41.532450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.532739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.533005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.534563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:41.536626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:41.537474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 15/35] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:41 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:41.540424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73438030-21e0-42fb-9298-9344130209dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:41.541004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:41.541477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73438030-21e0-42fb-9298-9344130209dc None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.542014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73438030-21e0-42fb-9298-9344130209dc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.542476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73438030-21e0-42fb-9298-9344130209dc None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.543350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73438030-21e0-42fb-9298-9344130209dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:41.546060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73438030-21e0-42fb-9298-9344130209dc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:41.602798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:41.645908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fd68-db85-45ba-8560-39ede67c480b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:41.678243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383ff8-23f7-4d09-82da-631d50dd42d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:41.678820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383ff8-23f7-4d09-82da-631d50dd42d9 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.679104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383ff8-23f7-4d09-82da-631d50dd42d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.679382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383ff8-23f7-4d09-82da-631d50dd42d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.680039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383ff8-23f7-4d09-82da-631d50dd42d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:41.681433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383ff8-23f7-4d09-82da-631d50dd42d9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:41.797521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 21/36] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 10 22:51:41 2022] POST /identity/v3/roles => generated 225 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:41.799491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01618-6c6a-4eec-89cf-1caea415585a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.799984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01618-6c6a-4eec-89cf-1caea415585a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.800269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01618-6c6a-4eec-89cf-1caea415585a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.802132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 22/37] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:41.803301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283c9c-92ce-426d-bbaa-4fef3e7d4d15 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.803594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283c9c-92ce-426d-bbaa-4fef3e7d4d15 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.803856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283c9c-92ce-426d-bbaa-4fef3e7d4d15 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.805280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 23/38] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:41.805407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 16/39] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 10 22:51:41 2022] POST /identity/v3/roles => generated 219 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:41.805883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283c9c-92ce-426d-bbaa-4fef3e7d4d15 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:41.811274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93425ed-1d28-4dfa-9e27-0c6e595e60b1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.811672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93425ed-1d28-4dfa-9e27-0c6e595e60b1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.816713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93425ed-1d28-4dfa-9e27-0c6e595e60b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.834986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d93425ed-1d28-4dfa-9e27-0c6e595e60b1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:41.837064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d93425ed-1d28-4dfa-9e27-0c6e595e60b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:41.929809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 17/40] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:51:41.931449 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:41.931814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:41.932162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:41.953162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:41.955030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:41.982633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283c9c-92ce-426d-bbaa-4fef3e7d4d15 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:42.005558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:42.032892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:42.034079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:42.034367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:42.034641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:42.036256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:42.037822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:42.038360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 18/41] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:42.039698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:42.039992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 24/42] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:42.046051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:42.060308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:42.147864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d4fe9-9c3a-46bf-8d49-f21ab31a7484 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:42.154115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568637-d048-4a70-a18a-7f027e26497b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:42.183206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:42.183856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:42.184144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:42.184424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:42.185549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:42.187106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:42.188697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 19/43] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:42.189452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:42.192271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:42.193198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:42.193488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:42.193765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] PATH_INFO: `/v3/domains/default/users/e793eb6ffc5c41a282b74540f29c0a31/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:42.196109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e793eb6ffc5c41a282b74540f29c0a31, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:42.197890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:42.233876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:42.287351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc21912-63f7-4c76-af53-51c7d5ca28bd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:42.319740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9009d-1a85-4dd5-83e9-ed14ff4359cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:42.320344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9009d-1a85-4dd5-83e9-ed14ff4359cf None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:42.320630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9009d-1a85-4dd5-83e9-ed14ff4359cf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:42.320907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9009d-1a85-4dd5-83e9-ed14ff4359cf None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:42.321567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9009d-1a85-4dd5-83e9-ed14ff4359cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:42.322994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9009d-1a85-4dd5-83e9-ed14ff4359cf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:42.630010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 25/44] 10.222.0.78 () {58 vars in 1361 bytes} [Wed Aug 10 22:51:42 2022] PUT /identity/v3/domains/default/users/e793eb6ffc5c41a282b74540f29c0a31/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:51:42.630974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:42.736105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785ca0a-0bbc-494f-b0a3-ff2d03f1ddfa None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:42.759596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 20/45] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 10 22:51:42 2022] POST /identity/v3/roles => generated 223 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:42.777728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d43d-bcdb-45db-b315-f06e61c4e578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:42.777728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d43d-bcdb-45db-b315-f06e61c4e578 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:42.778039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d43d-bcdb-45db-b315-f06e61c4e578 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:42.778281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d43d-bcdb-45db-b315-f06e61c4e578 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:42.779194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d43d-bcdb-45db-b315-f06e61c4e578 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:42.780568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d43d-bcdb-45db-b315-f06e61c4e578 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:42.933965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:43.005232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 26/46] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:51:42 2022] POST /identity/v3/projects => generated 312 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:44.646950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f131c-7b9d-403e-97d0-feb8f593a0d1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:44.647490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f131c-7b9d-403e-97d0-feb8f593a0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:44.647490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f131c-7b9d-403e-97d0-feb8f593a0d1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:44.648829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 21/47] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:44.654162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:44.654557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:44.654745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:44.673330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:44.674892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:44.740692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 27/48] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:44.763692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f131c-7b9d-403e-97d0-feb8f593a0d1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:44.875597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f131c-7b9d-403e-97d0-feb8f593a0d1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:44.877949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:44.918561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ec720-f1e4-4ec0-b0da-17ece75d5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:44.919217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ec720-f1e4-4ec0-b0da-17ece75d5a8b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:44.919499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ec720-f1e4-4ec0-b0da-17ece75d5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:44.919775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ec720-f1e4-4ec0-b0da-17ece75d5a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:44.929999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ec720-f1e4-4ec0-b0da-17ece75d5a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-128ec720-f1e4-4ec0-b0da-17ece75d5a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:44.935225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:44.936260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:44.937055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:44.937683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 22/49] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:44.943416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:44.991537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:45.012584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c78d8-4aa5-4d24-8ce3-049181981616 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.012922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c78d8-4aa5-4d24-8ce3-049181981616 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.013207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c78d8-4aa5-4d24-8ce3-049181981616 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.014544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 23/50] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:51:45.020005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.020294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.020574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.038609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:45.039852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:45.040793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4242fbc-83d5-451f-86e2-434e669bdcc8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:45.076439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:45.077039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.077317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.077585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.078234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:45.078531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:45.079989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:45.088172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 28/51] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:45.093665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:45.106480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 24/52] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:51:45.129610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:45.192340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad35e6e-c5a2-4f0f-9e38-4715d9052242 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:45.204792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:45.218757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:45.226381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:45.227065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.227345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.227616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.227815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d493a4-72fd-42a6-b707-0cac1b6f41b2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:45.228758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:45.230203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:45.231805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 29/53] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:45 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:45.237189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:45.261072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:45.261614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.261898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.262163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.263307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:45.264972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:45.266560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 25/54] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:45.272678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:45.337370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249434d-43d1-46c4-abdf-9bdc345a4f15 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:45.370631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8761f7-7724-46df-bafb-c83c671bb266 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:45.370885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:45.371766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.371884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.372058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.373184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:45.374685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:45.376248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 30/55] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:45.382754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:45.406046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20e3ac-b4df-47db-bf49-05d446c113d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:45.406673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20e3ac-b4df-47db-bf49-05d446c113d1 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.406961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20e3ac-b4df-47db-bf49-05d446c113d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.407244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20e3ac-b4df-47db-bf49-05d446c113d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.407892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20e3ac-b4df-47db-bf49-05d446c113d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:45.409371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20e3ac-b4df-47db-bf49-05d446c113d1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:45.431994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:45.481536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cb800-bfdc-4512-a56a-d1ea9202d421 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:45.514324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:45.514899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:45.515183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:45.515458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:45.516176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.domain.id=default, user.id=e793eb6ffc5c41a282b74540f29c0a31) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:45.516485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:45.517974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a282-6ed7-4f5b-8ce0-5a21cda7375a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:45.525559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 31/56] 10.222.0.78 () {56 vars in 1322 bytes} [Wed Aug 10 22:51:45 2022] GET /identity/v3/role_assignments?user.id=e793eb6ffc5c41a282b74540f29c0a31&scope.domain.id=default&role.id=be98a6e301f144d88c992f65bca561fb => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:45.535094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 26/57] 10.222.0.78 () {60 vars in 1140 bytes} [Wed Aug 10 22:51:45 2022] POST /identity/v3/projects => generated 298 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:51:47.570816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3d003-bdb1-4976-9a84-1e8726039476 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:47.571355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3d003-bdb1-4976-9a84-1e8726039476 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:47.571355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3d003-bdb1-4976-9a84-1e8726039476 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:47.572697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 32/58] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:47.577836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:47.578216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:47.578393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:47.595926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:47.597111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:47.604208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:47.668344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 27/59] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:47.693395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3d003-bdb1-4976-9a84-1e8726039476 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:47.790826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3d003-bdb1-4976-9a84-1e8726039476 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:47.803391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:47.824012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc242ea-24e2-4e2c-9b62-1c36aee1a4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:47.824631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc242ea-24e2-4e2c-9b62-1c36aee1a4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:47.824906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc242ea-24e2-4e2c-9b62-1c36aee1a4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:47.825172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc242ea-24e2-4e2c-9b62-1c36aee1a4a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:47.826256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc242ea-24e2-4e2c-9b62-1c36aee1a4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:47.827713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc242ea-24e2-4e2c-9b62-1c36aee1a4a9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:47.829246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 33/60] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:47.834983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:47.934198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8ba98-78e9-4f51-ad9b-bb8fa5cc81f6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:47.967397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097c019-6de5-47f0-adb9-8d0b3bf730a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:47.967993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097c019-6de5-47f0-adb9-8d0b3bf730a2 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:47.968275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097c019-6de5-47f0-adb9-8d0b3bf730a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:47.968550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097c019-6de5-47f0-adb9-8d0b3bf730a2 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:47.969209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097c019-6de5-47f0-adb9-8d0b3bf730a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:47.971233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097c019-6de5-47f0-adb9-8d0b3bf730a2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:48.122947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:48.128759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 28/61] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:51:47 2022] POST /identity/v3/users => generated 275 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:51.184109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7975e-1ffb-4b1a-826a-69e33719f231 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.184646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7975e-1ffb-4b1a-826a-69e33719f231 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.184646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7975e-1ffb-4b1a-826a-69e33719f231 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.185999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 34/62] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:51.195352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c97ca-15e2-4378-a923-630177611e07 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.201036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c97ca-15e2-4378-a923-630177611e07 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.201036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c97ca-15e2-4378-a923-630177611e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.223736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ca2c97ca-15e2-4378-a923-630177611e07 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:51.224977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ca2c97ca-15e2-4378-a923-630177611e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:51.312356 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 29/63] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:51.347793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7975e-1ffb-4b1a-826a-69e33719f231 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:51.473584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240aa8-5dab-4dd9-a14e-b598613eee39 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.473877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240aa8-5dab-4dd9-a14e-b598613eee39 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.474069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240aa8-5dab-4dd9-a14e-b598613eee39 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.475693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 30/64] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:51.485388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069e97-94cc-498b-9ad7-8bc69514e563 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.486249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069e97-94cc-498b-9ad7-8bc69514e563 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.486461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069e97-94cc-498b-9ad7-8bc69514e563 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.487995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 31/65] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:51:51.489073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.489360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.489637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.494953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7975e-1ffb-4b1a-826a-69e33719f231 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:51.511796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:51.513311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:51.540609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4cf1-db10-42e4-84ec-9f011332686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:51.541239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4cf1-db10-42e4-84ec-9f011332686b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.541516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4cf1-db10-42e4-84ec-9f011332686b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.541784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4cf1-db10-42e4-84ec-9f011332686b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.556177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a4cf1-db10-42e4-84ec-9f011332686b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3e3a4cf1-db10-42e4-84ec-9f011332686b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:51.562369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:51.563401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:51:51.564769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 35/66] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:51.565419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.565419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.565886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.585859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:51.587172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:51.606175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 32/67] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:51.611138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:51.641151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:51.678124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 36/68] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:51.679067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:51.688204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:51.733127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ef915-0b57-4da7-95e6-b64b1f1c41c5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:51.769632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:51.770580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.771079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.771562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.772469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:51.772990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:51.774774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:51.784494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 33/69] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:51:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:51.785579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:51.805941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45baa0-0540-44df-bf77-fc225b6209f1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:51.850054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:51.851137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.851533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.851813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.864450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:51.869190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:51.870051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:51:51.870847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 37/70] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:51.871653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:51.895375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbd401-cb5c-4869-95b1-a1001dc26586 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:51.903942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:51.920364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:51.934746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611d284-f972-4595-abee-9f3bfcb964a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:51.935375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d284-f972-4595-abee-9f3bfcb964a1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.935659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d284-f972-4595-abee-9f3bfcb964a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.935936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d284-f972-4595-abee-9f3bfcb964a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.946163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611d284-f972-4595-abee-9f3bfcb964a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2611d284-f972-4595-abee-9f3bfcb964a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:51.950871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:51.951763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:51.952527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 34/71] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:51:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:51.954439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d3fee-2ebf-4380-9c34-5b92a124162c None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:51.955481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d3fee-2ebf-4380-9c34-5b92a124162c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:51.955762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d3fee-2ebf-4380-9c34-5b92a124162c None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:51.957145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 35/72] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:51.959463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d3fee-2ebf-4380-9c34-5b92a124162c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:51.974960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256679a9-94f8-4311-880b-e089d61be470 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.009298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa50e-d92f-437a-b748-8c44ed2cb386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.009838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa50e-d92f-437a-b748-8c44ed2cb386 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.010117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa50e-d92f-437a-b748-8c44ed2cb386 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.010390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa50e-d92f-437a-b748-8c44ed2cb386 None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.032399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fa50e-d92f-437a-b748-8c44ed2cb386 None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.034600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fa50e-d92f-437a-b748-8c44ed2cb386 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.036353 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 38/73] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:51 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.037559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3f9e7-be27-4fd2-9e8b-b9072769316c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.037795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3f9e7-be27-4fd2-9e8b-b9072769316c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.038068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3f9e7-be27-4fd2-9e8b-b9072769316c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.043278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 39/74] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:52.043278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3f9e7-be27-4fd2-9e8b-b9072769316c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.066465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d3fee-2ebf-4380-9c34-5b92a124162c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.100325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.100917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.101203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.101475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.102125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:52.102421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.103833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e03b2e-97b5-48e3-8fbe-e90e8a294722 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.112997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 36/75] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.114120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193d512-2fa0-4ecc-b780-f36b50e67dde None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.114396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193d512-2fa0-4ecc-b780-f36b50e67dde None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.114679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193d512-2fa0-4ecc-b780-f36b50e67dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.131488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.131625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3193d512-2fa0-4ecc-b780-f36b50e67dde None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:52.132698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3193d512-2fa0-4ecc-b780-f36b50e67dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:52.144433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3f9e7-be27-4fd2-9e8b-b9072769316c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.166855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.177686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.178300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.178581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.178890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.179552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:52.179857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.181219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.189646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 40/76] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.190454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.199557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 37/77] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:52.199557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.199557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.199557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.226186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:52.227387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:52.290294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9512723-e89f-4899-be8b-1b9f242bace3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.296887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 38/78] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:52.296887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.325636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.326243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.326526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.326822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.346188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.395185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.395599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93004-1006-4aa4-9fa0-458ea9f08e2d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.396661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.397097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.429794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.430421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.430720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.430993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.432106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.433545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.435195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 39/79] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.435944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.495024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 41/80] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:51:52.495868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.535324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f970e4-6b09-4a55-9fd3-f248e6dfa6f9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.547445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.569300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.577328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.577328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.577328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.577328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.577328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.577328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 40/81] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.577717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.596104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05d6d-6b41-4f6f-a2d9-5cb8b0e7247d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.617351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.630281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.634432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.634432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.634432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.634432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.634432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.635738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 42/82] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.636777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.675886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5e5a6-4749-40ce-a77b-a34c23bb880f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.709426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.710044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.710318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.710588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.720669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:52.725317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:52.726357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 41/83] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:52.732179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.736676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f1f41-74c5-4006-bc6a-689426ec2fff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.754906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.770713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.771359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.771628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.771894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.827155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.830778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf09115-77a2-442c-a8c5-d8d3122186ba None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:52.837617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.851655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.862913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:52.863529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:52.863806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:52.864078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:52.865262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:52.866740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:52.868382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 42/84] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:52.869146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.937600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 43/85] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:51:52.938425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:52.958595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:52.985459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ff9f4-7026-412f-9a42-93d3a7000530 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.020314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.020916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.021199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.021480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.039284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a9d38-938d-4a0a-96d0-addd6cfa73e7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.054740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.073451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.074050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.074327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.074599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.075275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.076738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.088546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.089963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.185823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 43/86] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:51:53.186687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.231270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 44/87] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:51:52 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:53.232097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.253019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.291713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfa0fb-7bec-4321-97b6-a383b4df1c97 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.325133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.338704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.339621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.340403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.340680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.341366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:53.343810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.343810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.361140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 44/88] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:53.361140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.364113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e6f35-1e3a-4465-9bac-6762d9032e1a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.407016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.408188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.408482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.408765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.409654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=5d64a6dd6a5e42d0906f4f83111df269) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:53.409842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.412355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.419138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 45/89] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/role_assignments?user.id=5d64a6dd6a5e42d0906f4f83111df269&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:53.420262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.492085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc03763-cfa8-4979-80f9-332b19d0c65c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.531083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.547071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.547729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.548017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.555167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.555167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.555167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.555167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 45/90] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:53.555167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.565824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000b035-ec67-4503-b7c4-6711d8542e2e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.603881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.604428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.604709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.604984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.606278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.607636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.609655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 46/91] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:53.610583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.632209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.672749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f5224-e853-4cf5-8055-1f369a63326e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.715742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.716433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.718113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.718113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.718285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.721888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.723561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 46/92] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:53.726023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.753320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb1a1-c5e5-4ef7-80da-a011fb75c9af None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.796270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.797027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.797314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.797591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.798957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=e793eb6ffc5c41a282b74540f29c0a31) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:53.799532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.801734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.804329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.809363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 47/93] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/role_assignments?user.id=e793eb6ffc5c41a282b74540f29c0a31&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:53.810074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:53.886094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fd399-b90f-4348-9315-fda832f07a6e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:53.962940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:53.963169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:53.963594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:53.963879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:53.964155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:53.964907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] RBAC: Adding query filter params (role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=5d64a6dd6a5e42d0906f4f83111df269) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:53.965219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:53.968540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:53.976095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 47/94] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/role_assignments?user.id=5d64a6dd6a5e42d0906f4f83111df269&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:54.039073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8d458-e07b-4b35-bd5d-f231c283e6a8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:54.115611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:54.116493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:54.116921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:54.117367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:54.224833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:54.226513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:54.327866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:54.355814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 48/95] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:51:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:51:54.362124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:54.495194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c76880-87fd-4714-ab51-06a87bf706b4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:54.522624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:54.534811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c6f25-f163-46ff-9362-1280e1b0dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:54.536088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c6f25-f163-46ff-9362-1280e1b0dfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:54.536088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c6f25-f163-46ff-9362-1280e1b0dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:54.536088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c6f25-f163-46ff-9362-1280e1b0dfa8 None admin] PATH_INFO: `/v3/projects/c5237e2272634cea8732ab041557f290` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:54.537210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c6f25-f163-46ff-9362-1280e1b0dfa8 None admin] RBAC: Authorizing `identity:get_project(project_id=c5237e2272634cea8732ab041557f290)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:54.538680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c6f25-f163-46ff-9362-1280e1b0dfa8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:54.540640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 48/96] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:54 2022] GET /identity/v3/projects/c5237e2272634cea8732ab041557f290 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:54.546249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:54.668854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac5daf-6be1-4425-aada-8bc865106223 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:54.709891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:54.710506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:54.710816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:54.711093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:54.711823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=c5237e2272634cea8732ab041557f290, user.id=5d64a6dd6a5e42d0906f4f83111df269) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:54.712134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:54.713575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026551-8193-431b-adf3-12e73373dcb6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:54.723509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 49/97] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:51:54 2022] GET /identity/v3/role_assignments?user.id=5d64a6dd6a5e42d0906f4f83111df269&scope.project.id=c5237e2272634cea8732ab041557f290&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:56.073620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e361-919a-4f74-8b50-e48fd6199dbf None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.074327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e361-919a-4f74-8b50-e48fd6199dbf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.074327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e361-919a-4f74-8b50-e48fd6199dbf None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.075705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 49/98] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:56.081223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.082062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.082544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.105490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:56.107051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:56.113809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f71fc-d716-45d8-a8fe-b184820237de None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.114451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f71fc-d716-45d8-a8fe-b184820237de None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.114968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f71fc-d716-45d8-a8fe-b184820237de None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.116492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 50/99] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:51:56.129387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.129954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.130465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.156415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:56.158361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:56.161932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:56.255186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 50/100] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:56.269229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 51/101] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:56.312233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:56.319739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:56.488918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fbe68-ac74-4e32-8e7c-843125256f65 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:56.520436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0382a-a54d-46a3-a8c9-0045329a2444 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:56.523731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:56.524435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.524779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.525114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.526285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:56.527925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:56.529569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 51/102] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:56.535432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:56.567185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:56.580266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abd4ad-57b7-4073-a368-ea6d7969bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:56.580916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abd4ad-57b7-4073-a368-ea6d7969bbfa None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.581197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abd4ad-57b7-4073-a368-ea6d7969bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.581472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abd4ad-57b7-4073-a368-ea6d7969bbfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.592037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abd4ad-57b7-4073-a368-ea6d7969bbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a9abd4ad-57b7-4073-a368-ea6d7969bbfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:56.597300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:56.598249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 52/103] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:56.602813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5857144-1f39-474d-85a8-b9c2a28a7a9f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.603795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5857144-1f39-474d-85a8-b9c2a28a7a9f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.603795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5857144-1f39-474d-85a8-b9c2a28a7a9f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.603795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 53/104] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:56.604413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:56.607899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5857144-1f39-474d-85a8-b9c2a28a7a9f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:56.729219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c838d-93c8-42a2-b437-1db800925453 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:56.771243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5857144-1f39-474d-85a8-b9c2a28a7a9f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:56.777215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1300de-24aa-4851-86d2-a99d0217c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:56.778020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1300de-24aa-4851-86d2-a99d0217c9c6 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.778382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1300de-24aa-4851-86d2-a99d0217c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.778774 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1300de-24aa-4851-86d2-a99d0217c9c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.779509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1300de-24aa-4851-86d2-a99d0217c9c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:56.781057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1300de-24aa-4851-86d2-a99d0217c9c6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:56.813361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:56.814622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.815004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.815353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.816087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:56.816479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:56.817955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b22ce-ef06-4220-847a-3999725b0feb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:56.831409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 54/105] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:51:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:51:56.831409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f1df8-b826-4bed-b1fd-4db8450fc73e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.831409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f1df8-b826-4bed-b1fd-4db8450fc73e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.831409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f1df8-b826-4bed-b1fd-4db8450fc73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.847735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ff9f1df8-b826-4bed-b1fd-4db8450fc73e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:56.848928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ff9f1df8-b826-4bed-b1fd-4db8450fc73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:56.895222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:56.909626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 52/106] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:51:56 2022] POST /identity/v3/projects => generated 302 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:51:56.918452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c537-c849-46eb-abbc-fc874b20106f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.919339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c537-c849-46eb-abbc-fc874b20106f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.920819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c537-c849-46eb-abbc-fc874b20106f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.924084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 53/107] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:56.930780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4c537-c849-46eb-abbc-fc874b20106f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:56.942930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 55/108] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:56.944466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:56.945898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:56.946945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:56.983117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:56.985955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:57.156058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:57.174526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 56/109] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:57.174526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.187124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4c537-c849-46eb-abbc-fc874b20106f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.229229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.229875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.230157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.230421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.231742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:57.233918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:57.236631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 54/110] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:56 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:57.237593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.322793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001dad4-3261-463b-8bba-79e6fbc3b44e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.335867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:57.366212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2171d-de0f-480f-8118-ce6be34e9680 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.403411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.403962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.404240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.404507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.415160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:57.415729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e131e4-e66f-4f66-9a55-933b3cd232c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.416580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e131e4-e66f-4f66-9a55-933b3cd232c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.417054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e131e4-e66f-4f66-9a55-933b3cd232c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.417525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e131e4-e66f-4f66-9a55-933b3cd232c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:57.425702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:57.426541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:51:57.427355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:57.432037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 57/111] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:57.432722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.435243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e131e4-e66f-4f66-9a55-933b3cd232c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e2e131e4-e66f-4f66-9a55-933b3cd232c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:57.442424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:57.443313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:51:57.444128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:51:57.446691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 55/112] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:57.449223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24fe5d-9555-449c-bbbc-5fb75d633b5f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.449696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24fe5d-9555-449c-bbbc-5fb75d633b5f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.450132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24fe5d-9555-449c-bbbc-5fb75d633b5f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.451724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 56/113] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:57.452973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24fe5d-9555-449c-bbbc-5fb75d633b5f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.472348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:57.562094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8315182-8981-48b4-a6a9-a6a0580b5c3b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.600295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24fe5d-9555-449c-bbbc-5fb75d633b5f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.600886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.601526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.601810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.602090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.603319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:57.604883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:57.606522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 58/114] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:57.607392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.647125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.647742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.648024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.648300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.648972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:57.651032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:57.652431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83613561-124e-4c6f-98f6-d7e485b9d481 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:57.658040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:57.664767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 57/115] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:57.665966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.666256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.666592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.688352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:57.690562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:57.738053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f17d3-aa3c-4fcd-a080-42475e5ea16f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.771350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 58/116] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:57.772191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.775596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:57.782198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.783064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.790018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.790018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.790018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:57.790018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:57.790018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:57.797864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 59/117] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:57.798980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:57.883604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c81eea-93dd-47f2-966d-5844d5493304 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.905135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:57.909791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8650-f534-4907-88d9-61f150f600ca None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:57.916395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.917059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.917335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.919069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/5d64a6dd6a5e42d0906f4f83111df269/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.919616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] RBAC: Authorizing `identity:create_grant(project_id=507e4cf5a05041debeb551f9c60aa218, user_id=5d64a6dd6a5e42d0906f4f83111df269, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:57.921077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:57.944757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8544b1-84a6-473e-a65d-9266e9c55491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:57.945387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8544b1-84a6-473e-a65d-9266e9c55491 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.950187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8544b1-84a6-473e-a65d-9266e9c55491 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.950187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8544b1-84a6-473e-a65d-9266e9c55491 None admin] PATH_INFO: `/v3/users/75f03e1a002146f2bba85df6b348723b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.960995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8544b1-84a6-473e-a65d-9266e9c55491 None admin] RBAC: Authorizing `identity:get_user(user_id=75f03e1a002146f2bba85df6b348723b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:57.962454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8544b1-84a6-473e-a65d-9266e9c55491 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:57.964110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 60/118] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/users/75f03e1a002146f2bba85df6b348723b => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:57.965308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed629-464a-434c-9d74-389c079bd9ea None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:57.965583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed629-464a-434c-9d74-389c079bd9ea None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:57.965857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed629-464a-434c-9d74-389c079bd9ea None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:57.967258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 61/119] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:57.968989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aed629-464a-434c-9d74-389c079bd9ea None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.071032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aed629-464a-434c-9d74-389c079bd9ea None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.077990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 59/120] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:51:57 2022] PUT /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/5d64a6dd6a5e42d0906f4f83111df269/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:51:58.079481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.114005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.115104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.115618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.116175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.135506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:58.142191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:58.143219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:51:58.143990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:58.146687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 62/121] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:51:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:58.147701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.176243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.179985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.235749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d846-207a-4cbd-beee-a2f3336d758a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.284637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6569a-8e4b-4b20-81f2-bc02fc0dc4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.285280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6569a-8e4b-4b20-81f2-bc02fc0dc4ee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.285560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6569a-8e4b-4b20-81f2-bc02fc0dc4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.285867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6569a-8e4b-4b20-81f2-bc02fc0dc4ee None admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.287058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6569a-8e4b-4b20-81f2-bc02fc0dc4ee None admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:58.288514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6569a-8e4b-4b20-81f2-bc02fc0dc4ee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:58.290264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 60/122] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:58.291342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.291621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.291905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.292846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3a31f-efdc-4b06-86c5-42bae98807a1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.315293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:58.316561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:58.339396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.340013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.340292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.340991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.396934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 61/123] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:58.397807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.453756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:58.456410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:58.475540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.480473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.515085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6cc56-e9bf-4d9b-8a9f-0427370fab42 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.552402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.553022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.553367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.553650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.554372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:58.554717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:58.556165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:58.565420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 62/124] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:58.566400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.577143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 63/125] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:51:58.577998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.677175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f310a9-930c-449c-ab9f-0dbbf31d307e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.688090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e7ea-3adb-4321-8565-cbe4953bf3d8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.701048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.707955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.713745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.715464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.715464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.715464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.716261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:58.717721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:58.719519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 63/126] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:58.720427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.733253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.733890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.734172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.734447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.748510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:51:58.754445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:51:58.756663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 64/127] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:51:58.758641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.840619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ccb74-4267-4c86-bb2b-0156b5078ad5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.866611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63bfd5-0618-403c-ba0c-a3ceedfe0e80 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:58.877519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.878166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.878451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.878771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.879921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:58.881378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:58.883344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 64/128] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:58.884210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.913289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:58.913924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:58.914204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:58.914479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:58.916180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:58.917958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:58.920628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 65/129] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:58.921865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:58.964618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:58.976910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.013843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa7b28-2037-4931-a028-260141e5e5dc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.037650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72a64-785c-43be-98b8-a76800d72416 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.066731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.067404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.067691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.067972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/e793eb6ffc5c41a282b74540f29c0a31/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.069979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] RBAC: Authorizing `identity:create_grant(project_id=507e4cf5a05041debeb551f9c60aa218, user_id=e793eb6ffc5c41a282b74540f29c0a31, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.071493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.081332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.081965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.082249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.082528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.083231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:59.083536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.084928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.093709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 66/130] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:51:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:59.094363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.202532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198263b-2a2f-4015-9c81-185d17f2e61b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.209731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.218953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 65/131] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:51:58 2022] PUT /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/e793eb6ffc5c41a282b74540f29c0a31/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:51:59.218953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.249690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.250723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.251402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.252135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.253905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.255657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.257984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 67/132] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:59 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:59.259129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.344195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf1d90-349a-451d-befb-7a515fcf7873 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.368957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.389769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.390775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.390992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6d061-9c0a-4044-8840-07fdfb870cfd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.391459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.392436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.393529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] RBAC: Adding query filter params (role.id=790025671695475db6aa2a39c8746e54, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=75f03e1a002146f2bba85df6b348723b) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:51:59.393842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.396235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fc9ae-9e99-4e13-97ab-749c7d2d78e1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.402537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 66/133] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:51:59 2022] GET /identity/v3/role_assignments?user.id=75f03e1a002146f2bba85df6b348723b&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=790025671695475db6aa2a39c8746e54 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:59.404092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e49f4-fffa-4db0-9934-571e790733bc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.405835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e49f4-fffa-4db0-9934-571e790733bc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.406271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e49f4-fffa-4db0-9934-571e790733bc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.408962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 67/134] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:51:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:51:59.410491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e49f4-fffa-4db0-9934-571e790733bc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.455148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb44c6d-8c33-4611-9728-3acd86165f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.455902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb44c6d-8c33-4611-9728-3acd86165f22 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.456187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb44c6d-8c33-4611-9728-3acd86165f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.456586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb44c6d-8c33-4611-9728-3acd86165f22 None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.457874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb44c6d-8c33-4611-9728-3acd86165f22 None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.459372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb44c6d-8c33-4611-9728-3acd86165f22 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.461068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 68/135] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:51:59 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:59.462274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.462556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.462855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.483840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:59.485086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:59.529923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.548651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e49f4-fffa-4db0-9934-571e790733bc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.588029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 69/136] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:51:59.588902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.622065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.622767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.623049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.623323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/5d64a6dd6a5e42d0906f4f83111df269/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.628339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] RBAC: Authorizing `identity:create_grant(project_id=507e4cf5a05041debeb551f9c60aa218, user_id=5d64a6dd6a5e42d0906f4f83111df269, role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.629797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.713804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fc4e8-3502-4869-a576-c30192fa1ac6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:59.740931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 68/137] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:51:59 2022] PUT /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/5d64a6dd6a5e42d0906f4f83111df269/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:51:59.741603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.776938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:59.777611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:59.777888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:59.778159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] PATH_INFO: `/v3/projects/c5237e2272634cea8732ab041557f290` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:59.779535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=c5237e2272634cea8732ab041557f290)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:59.781990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:59.789054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 70/138] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:51:59 2022] GET /identity/v3/projects/c5237e2272634cea8732ab041557f290 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:51:59.793684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:59.841203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.885868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.975833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce918740-25f5-4130-8b86-da99790ddf4a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:00.012919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e375e1-2ada-4071-887c-9ddef3367dd4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:00.027305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:00.027949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.028233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.028509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.035344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:00.036877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:00.038646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 69/139] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:51:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:00.044231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:00.062559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7072a-3d9c-41f5-8e85-5a86d2ff9ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:00.063312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7072a-3d9c-41f5-8e85-5a86d2ff9ffb None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.064125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7072a-3d9c-41f5-8e85-5a86d2ff9ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.064401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7072a-3d9c-41f5-8e85-5a86d2ff9ffb None admin] PATH_INFO: `/v3/projects/c5237e2272634cea8732ab041557f290/users/5d64a6dd6a5e42d0906f4f83111df269/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.068647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7072a-3d9c-41f5-8e85-5a86d2ff9ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=c5237e2272634cea8732ab041557f290, user_id=5d64a6dd6a5e42d0906f4f83111df269, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:00.070283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7072a-3d9c-41f5-8e85-5a86d2ff9ffb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:00.169949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 71/140] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:51:59 2022] PUT /identity/v3/projects/c5237e2272634cea8732ab041557f290/users/5d64a6dd6a5e42d0906f4f83111df269/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:00.194741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72b732-b4e8-4438-87ab-3b4321f9f9a1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:00.210183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:00.239043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:00.239862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.240111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.240395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.241156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:00.242625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:00.410190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926b469-3782-4a14-b62c-1bd6d9063b48 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.410919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926b469-3782-4a14-b62c-1bd6d9063b48 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.410919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926b469-3782-4a14-b62c-1bd6d9063b48 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.412143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 72/141] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:00.418614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.418912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.419194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.438368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 70/142] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:00 2022] POST /identity/v3/users => generated 283 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:52:00.463267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:00.464593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:00.561121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 73/143] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:00.593706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:00.737813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc11ed-2252-4cdb-8ce9-37ad8911dcd0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:00.786951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:00.787608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.787889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.788162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.807249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:00.813241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:00.815821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:00.817750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 71/144] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:00.822687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:00.863971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:00.935798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d891459-e2e3-4a47-aa53-a60796af9a01 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:00.973782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:00.974388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:00.974681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:00.974946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:00.975600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:00.975895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:00.977258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:00.987294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 74/145] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:01.006915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:01.023757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:01.099983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778eda2-d13f-4b35-b79e-44225c64a75e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:01.142606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:01.143270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:01.143549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:01.143820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:01.144955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:01.146509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:01.148116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 72/146] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:00 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:01.153923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:01.256892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:01.266302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ddcf-09f8-4004-a86a-405ad735dd5b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:01.302823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:01.303428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:01.303702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:01.303968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:01.379373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:01.380843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:01.476789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:01.487076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 75/147] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:01.492972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:01.602140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:01.608641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13645-3627-401c-9e59-49cc4289d4ac None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:01.646175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f299cc6-4f07-4f60-87fc-962949ee05b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:01.646855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f299cc6-4f07-4f60-87fc-962949ee05b0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:01.647179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f299cc6-4f07-4f60-87fc-962949ee05b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:01.647450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f299cc6-4f07-4f60-87fc-962949ee05b0 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:01.648649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f299cc6-4f07-4f60-87fc-962949ee05b0 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:01.650440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f299cc6-4f07-4f60-87fc-962949ee05b0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:01.654464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 73/148] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:01 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:01.658485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:01.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944cc44-c4cf-476a-adc8-953c48fd7ff3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:01.799440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:01.800030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:01.800299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:01.800623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:01.801357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=5d64a6dd6a5e42d0906f4f83111df269) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:01.801659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:01.803323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67999847-7d11-4df9-b3c5-4b63143567f5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:01.809785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 76/149] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:01 2022] GET /identity/v3/role_assignments?user.id=5d64a6dd6a5e42d0906f4f83111df269&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:02.248600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba005ba8-adeb-471d-84d6-97a3c3aacdb2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.248997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba005ba8-adeb-471d-84d6-97a3c3aacdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.249169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba005ba8-adeb-471d-84d6-97a3c3aacdb2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.250522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 74/150] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:02.256773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79d69-0c84-46dd-a50a-4ed1243e26ba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.257066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79d69-0c84-46dd-a50a-4ed1243e26ba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.257342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79d69-0c84-46dd-a50a-4ed1243e26ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.287735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9ca79d69-0c84-46dd-a50a-4ed1243e26ba None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:02.288989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9ca79d69-0c84-46dd-a50a-4ed1243e26ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:02.350368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:02.372639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 77/151] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:02.400752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba005ba8-adeb-471d-84d6-97a3c3aacdb2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:02.488526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5eedd0-e2ff-4c77-8db9-92daa4aab3c9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.489150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5eedd0-e2ff-4c77-8db9-92daa4aab3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.489150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5eedd0-e2ff-4c77-8db9-92daa4aab3c9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.490404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 78/152] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:02.502555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.502847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.503042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.534799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:02.535996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:02.591170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba005ba8-adeb-471d-84d6-97a3c3aacdb2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:02.653904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:02.654583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.654892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.655200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.672094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 79/153] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:02.673691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:02.678713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:02.679572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:02.680719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 75/154] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:02.685748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:02.707324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:02.747254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:02.885008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614505d8-2697-4a58-87ad-a2db0caf1433 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:02.893761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c997e-9f60-4778-8b01-adec70ed0ad0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:02.933391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d37b9-6dbe-46d5-8820-92aca193300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:02.934048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:02.936338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d37b9-6dbe-46d5-8820-92aca193300e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.939916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d37b9-6dbe-46d5-8820-92aca193300e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.940434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d37b9-6dbe-46d5-8820-92aca193300e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.961491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d37b9-6dbe-46d5-8820-92aca193300e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8c7d37b9-6dbe-46d5-8820-92aca193300e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:02.968653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:02.970049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:02.970873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:02.974201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 76/155] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:02.977454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bfeba-4721-410e-a9f6-2181c32005ff None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.977735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bfeba-4721-410e-a9f6-2181c32005ff None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.978009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bfeba-4721-410e-a9f6-2181c32005ff None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.979272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:02.979873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:02.980860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:02.980860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:02.981123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:02.981342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:02.982725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100af60-5796-42f3-861d-8cfae4bf5aaf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:02.985234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 77/156] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:02.987699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bfeba-4721-410e-a9f6-2181c32005ff None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:03.003369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 80/157] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:03.004732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.005012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.005283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.043639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:03.044829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:03.152642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:03.178741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bfeba-4721-410e-a9f6-2181c32005ff None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:03.204620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 81/158] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:03.205490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:03.240035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:03.240651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.240978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.241243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.241908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:03.242253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:03.243830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:03.260795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 78/159] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:03.261891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:03.427290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afe4c-9173-488d-8a94-712a74b7a3b0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:03.437422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:03.467413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74476d-b33b-4595-acab-fe7f305acdfe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:03.487787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:03.488424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.488700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.489244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] PATH_INFO: `/v3/users/75f03e1a002146f2bba85df6b348723b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.491235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] RBAC: Authorizing `identity:get_user(user_id=75f03e1a002146f2bba85df6b348723b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:03.492687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:03.494262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 82/160] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:03 2022] GET /identity/v3/users/75f03e1a002146f2bba85df6b348723b => generated 289 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:03.495052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:03.537952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6af93c-4bd5-478e-888c-3a1946968e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:03.538576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6af93c-4bd5-478e-888c-3a1946968e0c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.538892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6af93c-4bd5-478e-888c-3a1946968e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.539161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6af93c-4bd5-478e-888c-3a1946968e0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.556377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6af93c-4bd5-478e-888c-3a1946968e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-af6af93c-4bd5-478e-888c-3a1946968e0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:03.561742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:03.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:03.564473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:03.572804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 79/161] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:03.578973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a5079-fc29-4078-beb5-742381433df4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.579898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a5079-fc29-4078-beb5-742381433df4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.580331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a5079-fc29-4078-beb5-742381433df4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.581949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 80/162] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:03.582803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a5079-fc29-4078-beb5-742381433df4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:03.599454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:03.689160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f5bb-202a-489d-b711-89bfb4bb20c4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:03.754684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a5079-fc29-4078-beb5-742381433df4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:03.758519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:03.759444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.759933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.760397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] PATH_INFO: `/v3/users/e793eb6ffc5c41a282b74540f29c0a31` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.762039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] RBAC: Authorizing `identity:get_user(user_id=e793eb6ffc5c41a282b74540f29c0a31)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:03.767722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:03.769614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 83/163] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:03 2022] GET /identity/v3/users/e793eb6ffc5c41a282b74540f29c0a31 => generated 247 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:03.770563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:03.825048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec7a50-551e-4910-a403-d771d41386b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:03.826979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7a50-551e-4910-a403-d771d41386b8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.828524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7a50-551e-4910-a403-d771d41386b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.830069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7a50-551e-4910-a403-d771d41386b8 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.832618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec7a50-551e-4910-a403-d771d41386b8 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:03.836092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec7a50-551e-4910-a403-d771d41386b8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:03.838250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 81/164] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:03 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:03.840102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:03.840630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:03.841095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:03.856517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:03.882590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:03.883829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:03.957769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d989e-e466-4f47-b1fd-c4aa330a7199 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:04.017114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 82/165] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:04.018810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:04.024484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:04.025091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.025562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:04.026427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.026724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.027404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:04.027706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:04.031421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:04.047127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 84/166] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:04.049516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:04.195653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915c81a-7655-40ea-ae75-6f7c2ce67836 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:04.228671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fa25f-3689-4ad0-843d-28ad95da41b9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:04.238460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:04.258321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:04.259011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.259290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.259560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.290811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:04.291663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.291944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.293118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/75f03e1a002146f2bba85df6b348723b/roles/790025671695475db6aa2a39c8746e54` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.298372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] RBAC: Authorizing `identity:create_grant(project_id=507e4cf5a05041debeb551f9c60aa218, user_id=75f03e1a002146f2bba85df6b348723b, role_id=790025671695475db6aa2a39c8746e54)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:04.300074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:04.351612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:04.353118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:04.356866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:04.422416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 85/167] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:04 2022] PUT /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218/users/75f03e1a002146f2bba85df6b348723b/roles/790025671695475db6aa2a39c8746e54 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:04.423510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:04.493415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 83/168] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:04.494448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:04.636135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c9f96-8225-4145-bfaf-558e00c26ccb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:04.683191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e20e6-4a6e-4922-a178-e697575aeda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:04.684427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e20e6-4a6e-4922-a178-e697575aeda7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.685104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e20e6-4a6e-4922-a178-e697575aeda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.685551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e20e6-4a6e-4922-a178-e697575aeda7 None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.687375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e20e6-4a6e-4922-a178-e697575aeda7 None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:04.689863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e20e6-4a6e-4922-a178-e697575aeda7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:04.690395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b31f4-8c6e-4a03-a660-4319ca882474 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:04.692764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 86/169] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:04 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:04.694277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704d9cb-b895-4f54-94d4-8b4ba4533671 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.694772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704d9cb-b895-4f54-94d4-8b4ba4533671 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.695217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704d9cb-b895-4f54-94d4-8b4ba4533671 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.695725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:04.697677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 87/170] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:04.698603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704d9cb-b895-4f54-94d4-8b4ba4533671 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:04.755797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde9ea1-3252-4d6d-9cc6-76c754d83304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:04.756336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde9ea1-3252-4d6d-9cc6-76c754d83304 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.756614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde9ea1-3252-4d6d-9cc6-76c754d83304 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.756884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde9ea1-3252-4d6d-9cc6-76c754d83304 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.758889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:04.784943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde9ea1-3252-4d6d-9cc6-76c754d83304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cdde9ea1-3252-4d6d-9cc6-76c754d83304 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:04.789711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:04.790852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:04.791716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:04.799019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 84/171] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:04.800386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.800667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.800938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.842248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:04.843615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:04.881562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704d9cb-b895-4f54-94d4-8b4ba4533671 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:04.929987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 85/172] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:04.938880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:04.940490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:04.941123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:04.941397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:04.941663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:04.943827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:04.945450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:04.947499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 88/173] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:04 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:04.948297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.050348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.057843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa40cf2-15da-4000-94f0-33fd4adb00f0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.058568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45b550-6da6-4f9e-b153-fb459f0a3c0c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.059387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.095351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.095959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.096239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.096509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.097506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:05.097840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.098403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.099273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaef86-c1c2-4175-9216-78b018b92444 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.099480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.099972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.100460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.108522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 89/174] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:05.114108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3cd72-7d52-4922-86ce-a8f43333f14a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.114399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3cd72-7d52-4922-86ce-a8f43333f14a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.114724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3cd72-7d52-4922-86ce-a8f43333f14a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.116195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 90/175] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:05.117027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810308-e779-4669-b818-2d858a701734 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.117357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810308-e779-4669-b818-2d858a701734 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.117647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810308-e779-4669-b818-2d858a701734 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.119058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 91/176] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:05.119761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5810308-e779-4669-b818-2d858a701734 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.183762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.185265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.233134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5810308-e779-4669-b818-2d858a701734 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.272526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.273124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.273399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.273665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.274384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=e793eb6ffc5c41a282b74540f29c0a31) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:05.274708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.276139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.282290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 92/177] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/role_assignments?user.id=e793eb6ffc5c41a282b74540f29c0a31&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:05.282905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.289866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.298860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 86/178] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:05.304246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.331274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.409840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939bf90-8a91-4f24-b389-a63771fe833c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.419352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa74ec-8ba1-4488-afa9-415eae6cff59 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.455518 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273858e-cc82-4d8d-bd60-687f6b4a30a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.456728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273858e-cc82-4d8d-bd60-687f6b4a30a4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.457007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273858e-cc82-4d8d-bd60-687f6b4a30a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.457271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273858e-cc82-4d8d-bd60-687f6b4a30a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.469703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273858e-cc82-4d8d-bd60-687f6b4a30a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:05.473578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3be5df-f259-4aa9-8adb-6ae711131a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.474467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be5df-f259-4aa9-8adb-6ae711131a94 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.475018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be5df-f259-4aa9-8adb-6ae711131a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.475509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be5df-f259-4aa9-8adb-6ae711131a94 None admin] PATH_INFO: `/v3/users/5d64a6dd6a5e42d0906f4f83111df269` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9273858e-cc82-4d8d-bd60-687f6b4a30a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:05.475685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:05.476536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 93/179] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:05.478797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.479535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.479535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.481320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3be5df-f259-4aa9-8adb-6ae711131a94 None admin] RBAC: Authorizing `identity:get_user(user_id=5d64a6dd6a5e42d0906f4f83111df269)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.482808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3be5df-f259-4aa9-8adb-6ae711131a94 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.489310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 87/180] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/users/5d64a6dd6a5e42d0906f4f83111df269 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:05.491827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.492108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.492380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.505639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:05.505816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:05.512949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:05.514727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:05.580478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 94/181] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:05.581546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.590999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.602142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.605298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 88/182] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:05.606962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.686144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bac69-28e1-4fb4-b8df-1e06f68e165b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.721458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.722088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.722360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.722626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.723799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.723927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d813ecb-3329-4eba-a0a2-c2d6fe200129 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.725321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.727079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 95/183] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:05.732506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.761224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.762114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.762538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.762975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.822800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.824330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:05.836143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.836143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.837356 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eedfd-1ddf-4753-8c16-5f79fe1697fb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:05.872869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:05.873468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:05.881621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:05.881621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:05.881621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:05.881621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:05.881621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:05.885421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 96/184] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:05.886108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.948990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 89/185] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:05.950058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:05.999506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1122ab-7c02-4c78-8be5-99f0b837d351 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.039676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.040369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.040653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.040925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.042087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.043572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.049505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 97/186] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:06.049505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.058130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.065428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.070861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172bf9-f1f7-4451-8d59-b081adcd21f4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.105176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.105963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.106241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.106512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.120035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:06.126115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:06.127292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:06.128114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 90/187] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:06.129372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.156757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6f089-056a-45e6-9bb0-e3494323d7ee None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.194541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.195423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.195701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.195973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.196689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] RBAC: Adding query filter params (role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=5d64a6dd6a5e42d0906f4f83111df269) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:06.196995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.199121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.214691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 98/188] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/role_assignments?user.id=5d64a6dd6a5e42d0906f4f83111df269&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:06.214691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.235739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3512b0d-c47a-4858-a7f7-df1bef9cc709 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.276288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.277267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.277639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.277974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] PATH_INFO: `/v3/users/bcb43926980247ac99222f83c3f0332f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.296860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] RBAC: Authorizing `identity:get_user(user_id=bcb43926980247ac99222f83c3f0332f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.299545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.301781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 91/189] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/users/bcb43926980247ac99222f83c3f0332f => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:06.302562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.318688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.336599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.363802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce2dc-19fa-4f76-a9c2-68da8230b284 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.426570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.433053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.433053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.433053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] PATH_INFO: `/v3/projects/c5237e2272634cea8732ab041557f290` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.433336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639034d-50fa-4feb-b0ff-121ed67e0acc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.436086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] RBAC: Authorizing `identity:get_project(project_id=c5237e2272634cea8732ab041557f290)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.437412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.440450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 99/190] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/projects/c5237e2272634cea8732ab041557f290 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:06.440450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.483692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.484653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.485660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.486158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.486835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.488310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.552824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5389-8cca-43a1-b1c4-f64519b3822d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.586421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.587058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.587340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.587608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.588265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:06.588562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.589983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.598445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 100/191] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:06.599241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.611494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.653327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 92/192] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:06 2022] POST /identity/v3/users => generated 297 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:06.654169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.705042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54694507-0080-4417-aced-1a9c5efdce4a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.737371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.744897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c243a8a-d98a-40ef-b19a-637839e43e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.745933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c243a8a-d98a-40ef-b19a-637839e43e30 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.745933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c243a8a-d98a-40ef-b19a-637839e43e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.746179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c243a8a-d98a-40ef-b19a-637839e43e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.777688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155dd8-06eb-483a-9e09-60ac8b07c34c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.819437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:06.819437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.819700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.819931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.820584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=c5237e2272634cea8732ab041557f290, user.id=5d64a6dd6a5e42d0906f4f83111df269) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:06.821936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.822325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.827054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c243a8a-d98a-40ef-b19a-637839e43e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:06.827054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c243a8a-d98a-40ef-b19a-637839e43e30 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:06.829560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 93/193] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/role_assignments?user.id=5d64a6dd6a5e42d0906f4f83111df269&scope.project.id=c5237e2272634cea8732ab041557f290&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:06.830383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:06.846895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:06.973478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767993d1-ee02-4db3-8f41-78d3f914c134 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:06.976493 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 101/194] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:06.977950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533d755-39b6-4c41-9ebe-4c32fa056111 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:06.978237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533d755-39b6-4c41-9ebe-4c32fa056111 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:06.978510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533d755-39b6-4c41-9ebe-4c32fa056111 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:06.984166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 102/195] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:06.985789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533d755-39b6-4c41-9ebe-4c32fa056111 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.014568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.021577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3507-6a24-4195-878d-7b30b7450dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.022792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3507-6a24-4195-878d-7b30b7450dba None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.023282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3507-6a24-4195-878d-7b30b7450dba None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.023757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3507-6a24-4195-878d-7b30b7450dba None admin] PATH_INFO: `/v3/users/bcb43926980247ac99222f83c3f0332f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.025854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d3507-6a24-4195-878d-7b30b7450dba None admin] RBAC: Authorizing `identity:get_user(user_id=bcb43926980247ac99222f83c3f0332f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.027823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d3507-6a24-4195-878d-7b30b7450dba None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.030988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 94/196] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/users/bcb43926980247ac99222f83c3f0332f => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:07.032489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700073f9-c1e4-411f-89af-308da40394db None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.032998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700073f9-c1e4-411f-89af-308da40394db None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.033489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700073f9-c1e4-411f-89af-308da40394db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.057665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-700073f9-c1e4-411f-89af-308da40394db None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:07.059195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-700073f9-c1e4-411f-89af-308da40394db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:07.136460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533d755-39b6-4c41-9ebe-4c32fa056111 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.150553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 95/197] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:07.151485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700073f9-c1e4-411f-89af-308da40394db None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.152276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.184692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.186381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.187175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.187646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.189909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.192565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.194485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 103/198] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:06 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:07.195460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.265367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700073f9-c1e4-411f-89af-308da40394db None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.294705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.304351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.304961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.305249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.305560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.338127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074447-57bf-4ff6-bae3-aa64e7f32da1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.375440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.376087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.376421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.376949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.387728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:07.392503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:07.393440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:07.394250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 104/199] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:07.395027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.395498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.397189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.446620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.523743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4ad93-a3d1-4f1f-8300-4c935b45744c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.532497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 96/200] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:07.533653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.565111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.570303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.570941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.571217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.571485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.572211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=bcb43926980247ac99222f83c3f0332f) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:07.572521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.574134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.581631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 105/201] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/role_assignments?user.id=bcb43926980247ac99222f83c3f0332f&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:07.582813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.714373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162d9e-b594-4e18-bcc5-ebf98e192119 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.777535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.778156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.778386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.778629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.779339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:07.779639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.780543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72c07d-03fd-4251-952a-8e3bd0314f0f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.781061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.783554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.794822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 97/202] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:07.800974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.824886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.825742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.826404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.826883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.828549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.830163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.838809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 106/203] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:07.842124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:07.945563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdb2a3-1a4d-4d6a-bb08-d7580073c111 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:07.980145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:07.983557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:07.984169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:07.984442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:07.984711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:07.985959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] RBAC: Adding query filter params (role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=bcb43926980247ac99222f83c3f0332f) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:07.986258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:07.987730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:07.997331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273d442-9ed2-4191-aa04-3993423f2aa7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:08.014887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 107/204] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/role_assignments?user.id=bcb43926980247ac99222f83c3f0332f&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:08.057569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:08.058228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:08.058501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:08.058790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] PATH_INFO: `/v3/users/75f03e1a002146f2bba85df6b348723b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:08.059953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] RBAC: Authorizing `identity:get_user(user_id=75f03e1a002146f2bba85df6b348723b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:08.061437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:08.063822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 98/205] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:07 2022] GET /identity/v3/users/75f03e1a002146f2bba85df6b348723b => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:08.069640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:08.121865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:08.214739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79b2d-8853-4cbe-b18f-c8e49b20b18b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:08.248666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:08.249273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:08.249547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:08.249815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:08.352729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:08.354541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:08.402772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:08.484007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 108/206] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:08.490272 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:08.663610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e56af-6537-4d3d-8002-39074b86359a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:08.716803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32fe12-6c06-4b53-b37e-0375e9aa5c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:08.717447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32fe12-6c06-4b53-b37e-0375e9aa5c6f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:08.717724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32fe12-6c06-4b53-b37e-0375e9aa5c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:08.717995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32fe12-6c06-4b53-b37e-0375e9aa5c6f None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:08.719176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32fe12-6c06-4b53-b37e-0375e9aa5c6f None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:08.720648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32fe12-6c06-4b53-b37e-0375e9aa5c6f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:08.722290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 99/207] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:08 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:08.728231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:08.840626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3c16-4ed3-4835-9ba3-effcced9abfb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:08.886807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:08.887417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:08.887697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:08.887965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:08.888696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] RBAC: Adding query filter params (role.id=790025671695475db6aa2a39c8746e54, scope.project.id=507e4cf5a05041debeb551f9c60aa218, user.id=75f03e1a002146f2bba85df6b348723b) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:08.889000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:08.890439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2e516-c971-4498-bd0f-f68ae28a5bbf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:08.892476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:08.897118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 109/208] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:08 2022] GET /identity/v3/role_assignments?user.id=75f03e1a002146f2bba85df6b348723b&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=790025671695475db6aa2a39c8746e54 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:09.143876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f45b5-522f-474e-8162-f007f0624f56 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.144462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f45b5-522f-474e-8162-f007f0624f56 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.144462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f45b5-522f-474e-8162-f007f0624f56 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.145939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 100/209] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:09.151595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac75e2-cf88-402d-98cc-b40093b28478 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.151881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac75e2-cf88-402d-98cc-b40093b28478 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.152161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac75e2-cf88-402d-98cc-b40093b28478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.173131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d2ac75e2-cf88-402d-98cc-b40093b28478 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:09.174392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d2ac75e2-cf88-402d-98cc-b40093b28478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:09.266942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 110/210] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:09.291702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f45b5-522f-474e-8162-f007f0624f56 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:09.323406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:09.327910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347321a-869c-400c-969a-08f49ce98a8f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.328187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347321a-869c-400c-969a-08f49ce98a8f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.328455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347321a-869c-400c-969a-08f49ce98a8f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.330571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 111/211] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:09.335809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.341415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.341415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.360203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:09.361736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:09.426401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f45b5-522f-474e-8162-f007f0624f56 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:09.446297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 112/212] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:09.471542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:09.473687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:09.474385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.474733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.475048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.486605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:09.492084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:09.493059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:09.493876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 101/213] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:09.500446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:09.559024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:09.599880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:09.618200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964dda4-1459-493a-8d1d-8afb0ae35c96 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:09.619797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e8de5-d0e9-4828-95fc-cfe718f74fe9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:09.655397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:09.656397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.656910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.657374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.658674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:09.660658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:09.662823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:09.667321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.667321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.667321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.667321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:09.667321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:09.667321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:09.668065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 113/214] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:09.669067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:09.681561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 102/215] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:09.683266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:09.777668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a52b0d-8bf5-4bc0-9cbe-21c4f8eab3af None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:09.806353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:09.816790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66d2b-52e7-470e-ac33-0409803f7d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:09.821784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4ddd9-bbfc-40e3-aea5-2a738a8bde54 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:09.821899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66d2b-52e7-470e-ac33-0409803f7d0c None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.821899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66d2b-52e7-470e-ac33-0409803f7d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.821899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66d2b-52e7-470e-ac33-0409803f7d0c None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.821899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66d2b-52e7-470e-ac33-0409803f7d0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:09.821899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66d2b-52e7-470e-ac33-0409803f7d0c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:09.854180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:09.854884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:09.855162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:09.855433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] PATH_INFO: `/v3/users/ccf5ad2661c248d78b91660b03bb914e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:09.869555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] RBAC: Authorizing `identity:get_user(user_id=ccf5ad2661c248d78b91660b03bb914e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:09.871073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:09.872846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 103/216] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:09 2022] GET /identity/v3/users/ccf5ad2661c248d78b91660b03bb914e => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:09.878889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:09.887893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:09.977021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadcf33-c806-42e0-8051-79431c05bf0d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:10.011909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:10.012523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.012798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.013066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.066693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 114/217] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:09 2022] POST /identity/v3/users => generated 297 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:10.086033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:10.087625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:10.134891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a464c0-5fa1-4a32-bfde-d0516095aeed None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.135210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a464c0-5fa1-4a32-bfde-d0516095aeed None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.135447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a464c0-5fa1-4a32-bfde-d0516095aeed None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.136874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 115/218] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:10.142709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.143326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.143800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.146364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:10.170275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:10.172184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:10.252884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 104/219] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:10.258533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:10.264104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 116/220] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:10.288464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:10.396867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d265e4d-21b1-453b-adbc-468a4e9df468 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:10.405160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:10.411963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535281c-8626-484a-9075-c66434d7a8bf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:10.434581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:10.445968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018d60-3f12-4eb8-9db7-7cc858d9443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:10.446974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018d60-3f12-4eb8-9db7-7cc858d9443a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.447539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018d60-3f12-4eb8-9db7-7cc858d9443a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.447815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018d60-3f12-4eb8-9db7-7cc858d9443a None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.449024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f018d60-3f12-4eb8-9db7-7cc858d9443a None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:10.450512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f018d60-3f12-4eb8-9db7-7cc858d9443a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:10.452428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 105/221] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:10.458406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981809a-f39d-4cad-88ae-2680eceff1dd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.458779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981809a-f39d-4cad-88ae-2680eceff1dd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.458883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac05e9c-c14c-4cd1-a562-23649984b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:10.459088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981809a-f39d-4cad-88ae-2680eceff1dd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.459790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac05e9c-c14c-4cd1-a562-23649984b472 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.460068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac05e9c-c14c-4cd1-a562-23649984b472 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.460346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 106/222] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:10.461067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac05e9c-c14c-4cd1-a562-23649984b472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.461182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981809a-f39d-4cad-88ae-2680eceff1dd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:10.474061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac05e9c-c14c-4cd1-a562-23649984b472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3ac05e9c-c14c-4cd1-a562-23649984b472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:10.479189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:10.483789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 117/223] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:10.484589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.485918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.485918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.504819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:10.505954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:10.588920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 118/224] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:10.594677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:10.614692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981809a-f39d-4cad-88ae-2680eceff1dd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:10.655300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:10.655938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.656266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.656557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.657407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=ccf5ad2661c248d78b91660b03bb914e) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:10.657714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:10.659363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:10.670686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 107/225] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/role_assignments?user.id=ccf5ad2661c248d78b91660b03bb914e&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:10.670686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:10.743175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:10.807039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61610399-ec5a-4b64-980c-dbaf2f891668 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:10.808644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:10.818113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df498cf-4da4-439a-85bc-49c3ab117e91 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:10.877280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:10.878036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:10.878501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.878703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:10.878979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.879250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.879587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:10.880201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:10.881054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:10.881527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:10.883282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:10.890844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:10.892904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 119/226] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:10.896528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:10.897446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:10.898243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 108/227] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:10.900553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:10.910990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:11.103533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:11.108029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012c3d-c8dc-4b1e-8a7c-d5ebd3a68860 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.112818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfadd-1475-490d-9c27-266678caef60 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.181253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.181903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.182112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.182373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.183254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:11.183553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.184919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.187624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.188786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.189154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.189544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] PATH_INFO: `/v3/users/bcb43926980247ac99222f83c3f0332f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.193142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] RBAC: Authorizing `identity:get_user(user_id=bcb43926980247ac99222f83c3f0332f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.193142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.193714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 120/228] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/users/bcb43926980247ac99222f83c3f0332f => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:11.202350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:11.209798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 109/229] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:11.218149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:11.263689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:11.373605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3568cd-c1f1-4ad0-9de8-1c99d3ecfe03 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.410200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.410874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.411156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.411430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] PATH_INFO: `/v3/users/bcb43926980247ac99222f83c3f0332f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.412592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] RBAC: Authorizing `identity:get_user(user_id=bcb43926980247ac99222f83c3f0332f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.414193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.415815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 110/230] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:11 2022] GET /identity/v3/users/bcb43926980247ac99222f83c3f0332f => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:11.432208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:11.450964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2a84e-35d6-4004-bfc9-f6a2d909b240 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.463058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:11.520876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.521512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.521792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.522063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.523238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.524678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.526606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 121/231] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:11 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:11.547745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:11.548551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec89ed-ab47-437e-bc02-eadb22592750 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.585523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.586504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.587157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.587680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.592177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.593886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.596763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 111/232] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:11 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:11.606116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:11.716256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3dd322-86fb-4421-bca8-2bcc86805f61 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.718065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:11.752205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640552c-d96a-4070-9138-e0ae838f031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.753277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640552c-d96a-4070-9138-e0ae838f031e None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.753736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640552c-d96a-4070-9138-e0ae838f031e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.754223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640552c-d96a-4070-9138-e0ae838f031e None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/bcb43926980247ac99222f83c3f0332f/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.754677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:11.757541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640552c-d96a-4070-9138-e0ae838f031e None admin] RBAC: Authorizing `identity:create_grant(project_id=6e772a239a88438d8a76221f9b346b3a, user_id=bcb43926980247ac99222f83c3f0332f, role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.759207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640552c-d96a-4070-9138-e0ae838f031e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.760544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8e0c-ffed-4d45-acdc-1d9a578f4b66 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:11.804379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfdd62-c8e8-49ec-8db5-7099fab3fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:11.805241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdd62-c8e8-49ec-8db5-7099fab3fdaa None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:11.805605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdd62-c8e8-49ec-8db5-7099fab3fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:11.805961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdd62-c8e8-49ec-8db5-7099fab3fdaa None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/bcb43926980247ac99222f83c3f0332f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:11.808179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfdd62-c8e8-49ec-8db5-7099fab3fdaa None admin] RBAC: Authorizing `identity:create_grant(project_id=6e772a239a88438d8a76221f9b346b3a, user_id=bcb43926980247ac99222f83c3f0332f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:11.809675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfdd62-c8e8-49ec-8db5-7099fab3fdaa None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:11.918157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 112/233] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:11 2022] PUT /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/bcb43926980247ac99222f83c3f0332f/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:11.968152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 122/234] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:11 2022] PUT /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/bcb43926980247ac99222f83c3f0332f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:12.429446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813d071-fe39-4c1a-8f52-13c32b756f87 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:12.429446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813d071-fe39-4c1a-8f52-13c32b756f87 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:12.429446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813d071-fe39-4c1a-8f52-13c32b756f87 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:12.429446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 113/235] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:12.434941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a93b2c-4251-4f14-bbf6-38f2c45a6778 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:12.435409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a93b2c-4251-4f14-bbf6-38f2c45a6778 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:12.435487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a93b2c-4251-4f14-bbf6-38f2c45a6778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:12.461305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-47a93b2c-4251-4f14-bbf6-38f2c45a6778 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:12.462496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-47a93b2c-4251-4f14-bbf6-38f2c45a6778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:12.556236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 123/236] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:12.580248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813d071-fe39-4c1a-8f52-13c32b756f87 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:12.666197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476216c-fb77-44f4-9a51-b773b2ac82d5 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:12.666481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476216c-fb77-44f4-9a51-b773b2ac82d5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:12.666796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476216c-fb77-44f4-9a51-b773b2ac82d5 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:12.668141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 124/237] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:12.679280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:12.679510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:12.679748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:12.714693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:12.716187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:12.744987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813d071-fe39-4c1a-8f52-13c32b756f87 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:12.820808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:12.821755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:12.822548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:12.823506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:12.824926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 125/238] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:12.831388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:12.833077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:12.834900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 114/239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:12.840524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:12.868380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:12.904065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:12.944086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:13.022328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa808086-7f47-4dd6-bba0-152f65ecad9f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.034543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abdacc-8a86-4d9a-b72a-c55458452cb3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.071885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb6e4f-b7b6-482c-ace7-01dc82836a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.072522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb6e4f-b7b6-482c-ace7-01dc82836a39 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.072795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb6e4f-b7b6-482c-ace7-01dc82836a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.073063 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb6e4f-b7b6-482c-ace7-01dc82836a39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.086710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb6e4f-b7b6-482c-ace7-01dc82836a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-becb6e4f-b7b6-482c-ace7-01dc82836a39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:13.092625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:13.093727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:13.094821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:13.102073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 115/240] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:13.102073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce0c2-5c13-42b7-bffa-1b2bd360766f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.102073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce0c2-5c13-42b7-bffa-1b2bd360766f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.102073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce0c2-5c13-42b7-bffa-1b2bd360766f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.102073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 116/241] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:13.109405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bce0c2-5c13-42b7-bffa-1b2bd360766f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.109497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323199e-6676-4446-9d8e-bfdc633b8e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.109497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323199e-6676-4446-9d8e-bfdc633b8e52 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.109497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323199e-6676-4446-9d8e-bfdc633b8e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.109497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323199e-6676-4446-9d8e-bfdc633b8e52 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.110341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323199e-6676-4446-9d8e-bfdc633b8e52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:13.111921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323199e-6676-4446-9d8e-bfdc633b8e52 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:13.203343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:13.219973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bce0c2-5c13-42b7-bffa-1b2bd360766f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.239983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 126/242] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:12 2022] POST /identity/v3/users => generated 293 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:13.241270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.241601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.241873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.266339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:13.267638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:13.276425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.277591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.278112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.278576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.289122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:13.289122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:13.289122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:13.305262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 117/243] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:13.311691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.356708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 127/244] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:13.388415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.398275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:13.487746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c925b4-7388-40eb-9293-cf76a94a014e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.527513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.528609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.529104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.529590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] PATH_INFO: `/v3/users/3ad13ea00e7d44f49673c96978e18332` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.538441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141500b-6765-4941-ac7d-ab5e5eb2816a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.544708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] RBAC: Authorizing `identity:get_user(user_id=3ad13ea00e7d44f49673c96978e18332)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:13.546515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:13.548838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 118/245] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/users/3ad13ea00e7d44f49673c96978e18332 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:13.554369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.559172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:13.581428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.582506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.583024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.583501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.595581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:13.600823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:13.601853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:13.602690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:13.605367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 128/246] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:13.613865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.683992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec310dc-bc3b-4998-af53-f2eba6b19968 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.726713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f9aff-f00b-4d90-9db3-baa80c543b5c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.728083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.729023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.729511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.730005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.771992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.772557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.772800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.773063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.773719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:13.774010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:13.775535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:13.784886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 129/247] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:13.791461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.817391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:13.819369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:13.920373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414fb0c-a6e3-4c22-ab2e-f868d16043bf None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:13.932745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 119/248] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:13.939204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:13.970872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:13.971491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:13.971768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:13.972033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] PATH_INFO: `/v3/users/ccf5ad2661c248d78b91660b03bb914e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:13.973667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] RBAC: Authorizing `identity:get_user(user_id=ccf5ad2661c248d78b91660b03bb914e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:13.975446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:13.977643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 130/249] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/users/ccf5ad2661c248d78b91660b03bb914e => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:13.983352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:14.075946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103764e8-2d13-461e-bf4c-6fc7263a1506 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:14.103959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.119761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:14.120772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.121276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.121747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.122990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a83fd7-42cc-4777-8d6d-6218f87cf0e8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:14.123375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:14.125816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:14.129321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 120/250] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:14.134311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.136627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:14.187523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:14.189120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.190298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.190970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.194087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:14.195798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:14.198393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 131/251] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:13 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:14.201538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.206051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:14.268408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.336289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.377650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436d9d-604b-4805-ac29-e0a2e809d273 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:14.384140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaef1b4-90ee-4df2-a200-c74e08c789a9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:14.441997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:14.443591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.444237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.445189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/ccf5ad2661c248d78b91660b03bb914e/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.448505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e772a239a88438d8a76221f9b346b3a, user_id=ccf5ad2661c248d78b91660b03bb914e, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:14.451161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:14.454590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:14.455468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.455955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.456435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.457400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] RBAC: Adding query filter params (role.id=790025671695475db6aa2a39c8746e54, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=3ad13ea00e7d44f49673c96978e18332) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:14.457925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:14.459897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d94e41-a7e1-46c3-ba07-9e7aaaa04a60 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:14.467578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 121/252] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:14 2022] GET /identity/v3/role_assignments?user.id=3ad13ea00e7d44f49673c96978e18332&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=790025671695475db6aa2a39c8746e54 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:14.515028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ed5eb-c524-4773-8fa4-2f2fb1eec296 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.516158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ed5eb-c524-4773-8fa4-2f2fb1eec296 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.516710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ed5eb-c524-4773-8fa4-2f2fb1eec296 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.518782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 122/253] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:14.524099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee622a-1195-4f13-98a7-b6749b75d874 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.524822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee622a-1195-4f13-98a7-b6749b75d874 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.525350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee622a-1195-4f13-98a7-b6749b75d874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.552513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d6ee622a-1195-4f13-98a7-b6749b75d874 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:14.554380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d6ee622a-1195-4f13-98a7-b6749b75d874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:14.603376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 132/254] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:14 2022] PUT /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/ccf5ad2661c248d78b91660b03bb914e/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:52:14.617024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.666813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 123/255] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:14.704171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:14.704928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eb6e-9756-480f-946b-d58e82931fa9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.705219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eb6e-9756-480f-946b-d58e82931fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.705492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eb6e-9756-480f-946b-d58e82931fa9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.707137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 124/256] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:14.712417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.713330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.713605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.743777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:14.744992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:14.845583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c3f3-5907-41f3-b628-146980153789 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:14.886704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:14.905410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:14.907909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:14.908184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:14.908450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:14.926254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 125/257] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:14.942752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:14.947485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:14.948386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:14.949185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 133/258] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:14.959786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:14.992472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:15.075616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:15.082083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37418da-654b-460a-a3c0-e20fe1dffe0a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.122156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.122800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.123075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.123340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:15.124006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:15.124301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:15.125714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:15.140679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 126/259] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:15.146567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:15.216228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6d82c-275b-423a-aec1-5ced94754bbc None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.283530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.284190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.284464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.284806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:15.301110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:15.308526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:15.309559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:15.310370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 134/260] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:15.317002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:15.363633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8575c-ae14-4c52-a1d5-217e7bb635c7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.420929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:15.449190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941b60-de24-4123-9251-78b245292894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.449848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941b60-de24-4123-9251-78b245292894 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.450124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941b60-de24-4123-9251-78b245292894 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.450394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941b60-de24-4123-9251-78b245292894 None admin] PATH_INFO: `/v3/users/bcb43926980247ac99222f83c3f0332f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:15.451666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa941b60-de24-4123-9251-78b245292894 None admin] RBAC: Authorizing `identity:get_user(user_id=bcb43926980247ac99222f83c3f0332f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:15.453222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa941b60-de24-4123-9251-78b245292894 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:15.455569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 127/261] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:15 2022] GET /identity/v3/users/bcb43926980247ac99222f83c3f0332f => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:15.461619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941b60-de24-4123-9251-78b245292894 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:15.497895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:15.545595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5892c6-f4a0-4f43-aba0-f48c13e5f0f1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.586424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941b60-de24-4123-9251-78b245292894 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.614598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.615927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.616501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.616994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:15.617916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:15.618425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:15.620244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:15.630095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.633150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.633437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.633708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:15.639813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 135/262] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:15.645690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:15.760091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acaa0c-f4a6-4758-ae6f-1ec9d67c32e9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.774994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:15.797196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.798325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.798809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.799086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] PATH_INFO: `/v3/users/bcb43926980247ac99222f83c3f0332f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:15.800643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] RBAC: Authorizing `identity:get_user(user_id=bcb43926980247ac99222f83c3f0332f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:15.802102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:15.803847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 136/263] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:15 2022] GET /identity/v3/users/bcb43926980247ac99222f83c3f0332f => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:15.809614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:15.839647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:15.841476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:15.891463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:15.926397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f1c4-47fe-49ca-8e27-e089ffb9056f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:15.970014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:15.971054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:15.971524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:15.972368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.027989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 128/264] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:16.035636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:16.067750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:16.118705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:16.120914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:16.261473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a15558-8fff-4719-ac41-b27b072d19dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:16.288527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:16.313445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 137/265] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:16.318962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:16.319115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:16.319617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.319892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.320158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.327368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:16.328862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:16.330885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 129/266] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:16 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:16.338064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:16.499862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ad8c6-4dcf-4f2d-8842-2105484684ad None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:16.543121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:16.560351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:16.560978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.561254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.561523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.562241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=bcb43926980247ac99222f83c3f0332f) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:16.562550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:16.564542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29cfd7-c7ad-492a-a41f-08428d2fc3c7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:16.579313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 130/267] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:16 2022] GET /identity/v3/role_assignments?user.id=bcb43926980247ac99222f83c3f0332f&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:16.581785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1b820-5c9a-4a42-a3ed-e1ae94ce1428 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:16.606868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368906b-a2a2-4218-8b58-0d69eaf011d0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.607390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368906b-a2a2-4218-8b58-0d69eaf011d0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.607390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368906b-a2a2-4218-8b58-0d69eaf011d0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.608734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 131/268] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:16.615716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.616000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.616274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.651937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:16.653670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:16.654807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248798b-2041-4091-9e19-31dca352194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:16.656138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248798b-2041-4091-9e19-31dca352194f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.656590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248798b-2041-4091-9e19-31dca352194f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.657003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248798b-2041-4091-9e19-31dca352194f None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.659335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248798b-2041-4091-9e19-31dca352194f None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:16.660956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248798b-2041-4091-9e19-31dca352194f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:16.662796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 138/269] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:16 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:16.668865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248798b-2041-4091-9e19-31dca352194f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:16.767514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:16.798882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 132/270] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:16.846905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:16.901745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248798b-2041-4091-9e19-31dca352194f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:16.966389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:16.967278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.967697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.968109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.969010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] RBAC: Adding query filter params (role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=bcb43926980247ac99222f83c3f0332f) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:16.969481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:16.971088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206a97f-1c19-42dd-bda0-d0469a0931cc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:16.977868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 139/271] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:16 2022] GET /identity/v3/role_assignments?user.id=bcb43926980247ac99222f83c3f0332f&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:16.979279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf00b6-901a-460a-9902-7ce2c7a11fc3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.980569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf00b6-901a-460a-9902-7ce2c7a11fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.981015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf00b6-901a-460a-9902-7ce2c7a11fc3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:16.982487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 140/272] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:16.989329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:16.990549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:16.991002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.008794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.011874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:17.013561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:17.030841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408dab0-5e03-410b-a3b9-f13165f0d38a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.111874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:17.113358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 141/273] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:17.114161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.114697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.115141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.132598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:17.137922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:17.139116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:17.140055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:17.142682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 133/274] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:17.148412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.153018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.247479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.301266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c436d-9c05-4a46-afe4-4da4297f6742 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.309359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a097ba-485e-46a6-b478-021904032686 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.340942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:17.341603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.341883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.342149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.342926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:17.343162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:17.345101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:17.347820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.355055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 142/275] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:17.362382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.363012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:17.363975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.364457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.365190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.367491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:17.369149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:17.371001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 134/276] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:17.377644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.495780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b48c7a-7b2f-4ed1-a33b-ebdb73ef6618 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.508869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e04ad-5776-489f-883d-145da4da0d02 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.527096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.530824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192919-a5ee-4d52-87a9-ab3c069fcd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:17.532048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192919-a5ee-4d52-87a9-ab3c069fcd82 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.532306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192919-a5ee-4d52-87a9-ab3c069fcd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.532577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192919-a5ee-4d52-87a9-ab3c069fcd82 None admin] PATH_INFO: `/v3/users/caf695b716e242808ec8c9c29be4f106` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.547846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab192919-a5ee-4d52-87a9-ab3c069fcd82 None admin] RBAC: Authorizing `identity:get_user(user_id=caf695b716e242808ec8c9c29be4f106)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:17.549298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab192919-a5ee-4d52-87a9-ab3c069fcd82 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:17.549397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35db891-5f49-4ca4-8b56-ba6fb8c4e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:17.549904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35db891-5f49-4ca4-8b56-ba6fb8c4e64a None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.550185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35db891-5f49-4ca4-8b56-ba6fb8c4e64a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.550455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35db891-5f49-4ca4-8b56-ba6fb8c4e64a None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.550947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 143/277] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:17 2022] GET /identity/v3/users/caf695b716e242808ec8c9c29be4f106 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:17.555894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35db891-5f49-4ca4-8b56-ba6fb8c4e64a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:17.556004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc4be8-bdca-478a-a5a4-874f832cbcec None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.556004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc4be8-bdca-478a-a5a4-874f832cbcec None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.556004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc4be8-bdca-478a-a5a4-874f832cbcec None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.556183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35db891-5f49-4ca4-8b56-ba6fb8c4e64a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:17.556249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 144/278] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:17.556579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc4be8-bdca-478a-a5a4-874f832cbcec None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.616051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.662759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc4be8-bdca-478a-a5a4-874f832cbcec None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.681943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 135/279] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:17 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:52:17.683896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.684365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.687711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.698923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:17.699586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.700160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.700440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.701240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, user.id=caf695b716e242808ec8c9c29be4f106, scope.system=all) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:17.701553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:17.703703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbbef-a45c-4710-8141-cedfdb5c19bd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:17.704850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:17.707279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:17.709552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 145/280] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 10 22:52:17 2022] GET /identity/v3/role_assignments?user.id=caf695b716e242808ec8c9c29be4f106&scope.system=all&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:17.711069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76ae7-0063-4ee5-87e6-935b070b8b63 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.711345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76ae7-0063-4ee5-87e6-935b070b8b63 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.711616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76ae7-0063-4ee5-87e6-935b070b8b63 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.713854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 146/281] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:17.720906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:17.721286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:17.721465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:17.741529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:17.742877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:17.805623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 136/282] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:17.827699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 147/283] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:17.829074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.851540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:17.913698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.953594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45b2e2-66d4-4e5b-a089-d74f5211260e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.958386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:17.980551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb8fd9-1ec4-4a23-9777-792248b6c83c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:17.999698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.000749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.001213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.001918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.014224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:18.019340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:18.020247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:18.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:18.026755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 137/284] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:18.028848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.029546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.029653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.029817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.030093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.043806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:18.049330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:18.059498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 148/285] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:18.060305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.160835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa877e-b1b2-4b40-9470-dbb07d30c745 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:18.193664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb95cdb-4f86-49b9-a33b-92458a723d2b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:18.206684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:18.224728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.225582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.226041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.226457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.227288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:18.227744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:18.229264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:18.230136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.230833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.231183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.234942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.234942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:18.234942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:18.234942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:18.244024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 149/286] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:18.246406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 138/287] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:18.249676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.253398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.298739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:18.453964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966d777-f823-4c51-856b-71cf060a5243 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:18.472900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7c43c-e6be-4f82-a713-b19288a5febb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:18.494099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.494770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.495818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.497734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] PATH_INFO: `/v3/users/3ad13ea00e7d44f49673c96978e18332` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.499703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] RBAC: Authorizing `identity:get_user(user_id=3ad13ea00e7d44f49673c96978e18332)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:18.502759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:18.506822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 139/288] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:18 2022] GET /identity/v3/users/3ad13ea00e7d44f49673c96978e18332 => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:18.510536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.537156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.537785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.538059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.538329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] PATH_INFO: `/v3/users/ccf5ad2661c248d78b91660b03bb914e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.540316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] RBAC: Authorizing `identity:get_user(user_id=ccf5ad2661c248d78b91660b03bb914e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:18.542620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:18.544467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 150/289] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:18 2022] GET /identity/v3/users/ccf5ad2661c248d78b91660b03bb914e => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:18.551878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.626641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:18.683455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:18.699220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced1398-d9d7-4dd1-89a9-c4f3af798947 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:18.742728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1726680-e280-4078-939d-0dafc51bb951 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:18.779610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.780219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.780494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.780764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.795380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:18.796031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:18.796314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:18.796580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:18.797786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:18.799272 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:18.801105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 140/290] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:18 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:18.808334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:18.883819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:18.885293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:18.966016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:19.001043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d6742-d0c5-4e11-baf4-a2d2c44a488b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:19.034618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 151/291] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:19.039897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:19.091549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e726e79-aed0-429b-b563-1c1aa896ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:19.092229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726e79-aed0-429b-b563-1c1aa896ea16 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:19.092508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726e79-aed0-429b-b563-1c1aa896ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:19.092829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726e79-aed0-429b-b563-1c1aa896ea16 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/3ad13ea00e7d44f49673c96978e18332/roles/790025671695475db6aa2a39c8746e54` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:19.096973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e726e79-aed0-429b-b563-1c1aa896ea16 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e772a239a88438d8a76221f9b346b3a, user_id=3ad13ea00e7d44f49673c96978e18332, role_id=790025671695475db6aa2a39c8746e54)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:19.098413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e726e79-aed0-429b-b563-1c1aa896ea16 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:19.168647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c730-411f-4b7d-99f0-0be8c084b411 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:19.204323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:19.205266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:19.205734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:19.206150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:19.207476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:19.209107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:19.211392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 152/292] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:19 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:19.218306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:19.228389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 141/293] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:18 2022] PUT /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a/users/3ad13ea00e7d44f49673c96978e18332/roles/790025671695475db6aa2a39c8746e54 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:19.252139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:19.346416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7ce3-5a02-400c-b4e2-873b278ea6ec None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:19.384997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:19.385979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:19.386394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:19.386830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:19.387699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=ccf5ad2661c248d78b91660b03bb914e) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:19.388172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:19.389850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ca302-e293-408e-9b27-60a779b13076 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:19.396899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 153/294] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:19 2022] GET /identity/v3/role_assignments?user.id=ccf5ad2661c248d78b91660b03bb914e&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:19.980364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7345b-80db-4a5d-9e5f-6bfd47de75d9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:19.980908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7345b-80db-4a5d-9e5f-6bfd47de75d9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:19.980908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7345b-80db-4a5d-9e5f-6bfd47de75d9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:19.982232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 142/295] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:19.987565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:19.987961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:19.988099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.007545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:20.008738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:20.079473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 154/296] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:20.101697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7345b-80db-4a5d-9e5f-6bfd47de75d9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.125062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.208874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7345b-80db-4a5d-9e5f-6bfd47de75d9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.244127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065e23f-be60-42e6-ae5f-cd6db19d377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.244732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e23f-be60-42e6-ae5f-cd6db19d377d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.245006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e23f-be60-42e6-ae5f-cd6db19d377d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.245322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065e23f-be60-42e6-ae5f-cd6db19d377d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.256091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065e23f-be60-42e6-ae5f-cd6db19d377d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a065e23f-be60-42e6-ae5f-cd6db19d377d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:20.261316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:20.262187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:20.263050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 143/297] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:20.269971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.282589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.284265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b362-e7a3-4b0b-8f8f-d4c26b7a40f3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.284548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b362-e7a3-4b0b-8f8f-d4c26b7a40f3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.284819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b362-e7a3-4b0b-8f8f-d4c26b7a40f3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.286306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 144/298] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:20.291139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44225f09-476b-40ab-b434-341edcc25f9e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.299622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44225f09-476b-40ab-b434-341edcc25f9e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.299622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44225f09-476b-40ab-b434-341edcc25f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.314026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-44225f09-476b-40ab-b434-341edcc25f9e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:20.315550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44225f09-476b-40ab-b434-341edcc25f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:20.387421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 145/299] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:20.388501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360499fd-07af-43b5-bb14-99e8b658ed57 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.388786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360499fd-07af-43b5-bb14-99e8b658ed57 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.389056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360499fd-07af-43b5-bb14-99e8b658ed57 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.390423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 146/300] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:20.391141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab302c7-840f-452b-8582-1aa6cabfa009 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.395891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.396177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.396449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.414625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:20.415836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:20.419553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.441040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.441638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.441910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.442173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.442874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:20.443211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:20.444672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:20.452785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 155/301] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:20.452785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.498735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 147/302] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:20.499510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.529126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.569243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53aee0-3f52-4d0d-9a40-45d32a0f746e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.605153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.605777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.606050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.606383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.607583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b636ccb-5c65-4848-9f73-8f3606177437 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.617140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:20.623034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:20.623916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:20.624806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 156/303] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:20.626087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.649374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.650019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.650302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.650568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] PATH_INFO: `/v3/users/caf695b716e242808ec8c9c29be4f106` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.651846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] RBAC: Authorizing `identity:get_user(user_id=caf695b716e242808ec8c9c29be4f106)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:20.653327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:20.655345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 148/304] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:20 2022] GET /identity/v3/users/caf695b716e242808ec8c9c29be4f106 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:20.656278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.670767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.739468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca8db3-3f69-43de-946d-f39d227d2cff None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.764252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.764621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e955e-0f07-485a-a4c0-ba2d2ee7a8eb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.774543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.775369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.775800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.776359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.777180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:20.778631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:20.780218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 157/305] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:20.786736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.801776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.802396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.802751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.802962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.803615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:20.803909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:20.805261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:20.817986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 149/306] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:20.817986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:20.888916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:20.896015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f58a8-391e-48dc-a2cc-6b50ff40920a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.920535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9011-def7-464c-b209-82a63399f5d2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:20.930263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.930962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.931247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.931521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] PATH_INFO: `/v3/system/users/caf695b716e242808ec8c9c29be4f106/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.932933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=caf695b716e242808ec8c9c29be4f106, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:20.934437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:20.955835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:20.956641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:20.957341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:20.957614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] PATH_INFO: `/v3/groups` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:20.966720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:20.966720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:21.075210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 158/307] 10.222.0.78 () {58 vars in 1334 bytes} [Wed Aug 10 22:52:20 2022] PUT /identity/v3/system/users/caf695b716e242808ec8c9c29be4f106/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:21.083298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:21.151155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 150/308] 10.222.0.78 () {60 vars in 1134 bytes} [Wed Aug 10 22:52:20 2022] POST /identity/v3/groups => generated 229 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:21.168514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:21.225304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c35f9-4a05-49e5-95cf-a4e083f45809 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:21.279704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69d2ae-0848-4889-8e22-f9ded668e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:21.280609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d2ae-0848-4889-8e22-f9ded668e56e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:21.280888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d2ae-0848-4889-8e22-f9ded668e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:21.281367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d2ae-0848-4889-8e22-f9ded668e56e None admin] PATH_INFO: `/v3/users/eaa5ef83578845dabc6f22cccc1bc919` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:21.297420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69d2ae-0848-4889-8e22-f9ded668e56e None admin] RBAC: Authorizing `identity:get_user(user_id=eaa5ef83578845dabc6f22cccc1bc919)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:21.299086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69d2ae-0848-4889-8e22-f9ded668e56e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:21.300821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 159/309] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:21 2022] GET /identity/v3/users/eaa5ef83578845dabc6f22cccc1bc919 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:21.306953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:21.420020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174bef9-f845-4ed1-a40d-f72bcc5d5d68 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:21.455431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:21.461996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:21.462811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:21.463237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:21.463675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:21.464544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] RBAC: Adding query filter params (role.id=790025671695475db6aa2a39c8746e54, user.id=eaa5ef83578845dabc6f22cccc1bc919, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:21.464998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:21.466584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6dfb2-726b-42e7-b9da-2f5b655cdb82 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:21.482169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 151/310] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 10 22:52:21 2022] GET /identity/v3/role_assignments?user.id=eaa5ef83578845dabc6f22cccc1bc919&scope.system=all&role.id=790025671695475db6aa2a39c8746e54 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:21.578342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1467cad-4619-4421-b076-ff27add7b4b7 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:21.578621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1467cad-4619-4421-b076-ff27add7b4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:21.578925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1467cad-4619-4421-b076-ff27add7b4b7 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:21.580274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 160/311] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:21.585960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:21.586235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:21.586500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:21.606176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:21.607415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:21.684307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 152/312] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:21.708591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1467cad-4619-4421-b076-ff27add7b4b7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:21.815749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1467cad-4619-4421-b076-ff27add7b4b7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:21.830432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:21.850120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:21.850768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:21.853192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:21.853192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:21.861856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:21.866711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:21.867820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:21.868845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:21.872644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 161/313] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:21.874008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:21.987256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e51b-2f07-4f85-876c-afbc0679cdf6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:22.027075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:22.045608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:22.046218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:22.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:22.047298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:22.047960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:22.048254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:22.049603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:22.066764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 153/314] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:22.073764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:22.178798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c997ee7-737c-41cd-8368-eb8c1bde30dd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:22.213701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:22.214317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:22.214586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:22.214879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] PATH_INFO: `/v3/users/3ad13ea00e7d44f49673c96978e18332` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:22.216097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] RBAC: Authorizing `identity:get_user(user_id=3ad13ea00e7d44f49673c96978e18332)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:22.217617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:22.224370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 162/315] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:22 2022] GET /identity/v3/users/3ad13ea00e7d44f49673c96978e18332 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:22.225750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:22.329010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf69eb-0973-4623-a323-ada1daf36c4c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:22.369785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:22.370407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:22.370701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:22.370975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:22.471001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:22.503875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:22.505636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:22.637643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 154/316] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:22.640755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:22.687072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:22.744096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34569-5156-4db6-8757-e88fdfd93a4c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:22.778463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b55ab-76f3-4e61-b46b-500898ec7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:22.779112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b55ab-76f3-4e61-b46b-500898ec7e92 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:22.779386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b55ab-76f3-4e61-b46b-500898ec7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:22.779656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b55ab-76f3-4e61-b46b-500898ec7e92 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:22.780761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b55ab-76f3-4e61-b46b-500898ec7e92 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:22.782290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b55ab-76f3-4e61-b46b-500898ec7e92 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:22.787727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 163/317] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:22 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:22.790383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:22.873678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:22.898845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf306-9de3-4380-b97a-2bdaa76fd8cf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:22.923507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:22.924378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:22.924920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:22.925357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:22.926231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] RBAC: Adding query filter params (role.id=790025671695475db6aa2a39c8746e54, scope.project.id=6e772a239a88438d8a76221f9b346b3a, user.id=3ad13ea00e7d44f49673c96978e18332) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:22.926726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:22.928308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acb3f8-b26d-4604-8f5f-ab617f1ecea6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:22.939020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 155/318] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:22 2022] GET /identity/v3/role_assignments?user.id=3ad13ea00e7d44f49673c96978e18332&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=790025671695475db6aa2a39c8746e54 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:23.300555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba7be5-fbbb-46e5-8c46-8639d77d88f9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.301040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba7be5-fbbb-46e5-8c46-8639d77d88f9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.301040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba7be5-fbbb-46e5-8c46-8639d77d88f9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.302496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 164/319] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:23.307443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2000a-fc4f-4776-b64d-75132da4bda1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.308284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2000a-fc4f-4776-b64d-75132da4bda1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.308284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2000a-fc4f-4776-b64d-75132da4bda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.326078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-15f2000a-fc4f-4776-b64d-75132da4bda1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:23.327281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-15f2000a-fc4f-4776-b64d-75132da4bda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:23.395435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 156/320] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:23.418003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba7be5-fbbb-46e5-8c46-8639d77d88f9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:23.528998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba7be5-fbbb-46e5-8c46-8639d77d88f9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:23.536093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37908f-3733-4867-94d8-3f80e1b2e6ad None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.536385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37908f-3733-4867-94d8-3f80e1b2e6ad None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.536654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37908f-3733-4867-94d8-3f80e1b2e6ad None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.537992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 157/321] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:23.540373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:23.543016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466077-cbf6-4c4e-9e62-915ed183a954 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.543301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466077-cbf6-4c4e-9e62-915ed183a954 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.543579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466077-cbf6-4c4e-9e62-915ed183a954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.565376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-47466077-cbf6-4c4e-9e62-915ed183a954 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:23.565376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-47466077-cbf6-4c4e-9e62-915ed183a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:23.565659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:23.565659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.565659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.565930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.576506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:23.581429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:23.582310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:23.583109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:23.584163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 165/322] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:23.590018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:23.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 158/323] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:23.642300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64fbbd-173d-4388-823f-f143e4e863ab None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.642606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64fbbd-173d-4388-823f-f143e4e863ab None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.642902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64fbbd-173d-4388-823f-f143e4e863ab None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.644123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 159/324] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:23.649338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.649607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.649875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.668190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:23.669405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:23.672798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:23.695420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b04e1-aab3-4df1-ac80-f7e71a76fdcb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:23.729173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:23.729822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.730118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.730391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.731068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:23.731364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:23.732727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:23.736437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 160/325] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:23.737293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:23.741702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 166/326] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:23.747638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:23.779904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:23.838069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9c1b5-bc61-4761-ac87-4ba31f0e4c02 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:23.846481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e2fd5-2c77-4506-b164-2b35839b9f4f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:23.870561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:23.871201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.871474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.871738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.872815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:23.874228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:23.876635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 161/327] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:23.877383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:23.882873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:23.883495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.883772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.884044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] PATH_INFO: `/v3/users/caf695b716e242808ec8c9c29be4f106` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.885158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] RBAC: Authorizing `identity:get_user(user_id=caf695b716e242808ec8c9c29be4f106)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:23.886600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:23.887624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:23.888498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 167/328] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:23 2022] GET /identity/v3/users/caf695b716e242808ec8c9c29be4f106 => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:23.888904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:23.978989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757421-923b-46d9-bfad-4b0bb3a598fe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:23.988186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:23.991265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094f96a-e81b-4671-a563-1500f67c9c88 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:24.013656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:24.014290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:24.014568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:24.014867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:24.023727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:24.024313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:24.024584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:24.024864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] PATH_INFO: `/v3/groups` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:24.025493 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:24.025663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:24.026943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:24.030458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:24.031995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:24.032800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 162/329] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:24.033905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:24.097955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:24.130246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef9b9-5c8e-4ad3-a944-52f87909a333 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:24.161726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 168/330] 10.222.0.78 () {60 vars in 1134 bytes} [Wed Aug 10 22:52:23 2022] POST /identity/v3/groups => generated 226 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:24.162118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:24.162592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:24.162753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:24.163036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:24.163307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:24.164303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] RBAC: Adding query filter params (role.id=f8c74cd9c7d245249553a39801f08d82, user.id=caf695b716e242808ec8c9c29be4f106, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:24.164775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:24.166289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:24.194688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 163/331] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 10 22:52:24 2022] GET /identity/v3/role_assignments?user.id=caf695b716e242808ec8c9c29be4f106&scope.system=all&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 510 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:24.284009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12668aef-4200-4361-9b65-f8f9fec5930c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:24.323693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:24.324386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:24.324571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:24.324840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:24.325511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:24.325886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:24.327352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:24.338406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 169/332] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:24.344580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:24.452585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58465d-b638-4fde-9d8f-c957870fa430 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:24.483583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:24.492914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361e7c4-eb16-489f-af89-9cbbf0810bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:24.493553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361e7c4-eb16-489f-af89-9cbbf0810bbe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:24.493834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361e7c4-eb16-489f-af89-9cbbf0810bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:24.494112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361e7c4-eb16-489f-af89-9cbbf0810bbe None admin] PATH_INFO: `/v3/users/eaa5ef83578845dabc6f22cccc1bc919` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:24.506760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361e7c4-eb16-489f-af89-9cbbf0810bbe None admin] RBAC: Authorizing `identity:get_user(user_id=eaa5ef83578845dabc6f22cccc1bc919)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:24.506760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361e7c4-eb16-489f-af89-9cbbf0810bbe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:24.506760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 164/333] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:24 2022] GET /identity/v3/users/eaa5ef83578845dabc6f22cccc1bc919 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:24.507013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:24.533779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:24.615143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690f781-7dec-4a8e-aace-cf34362d55ed None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:24.652070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354b6c5-765f-480b-8ab5-fe12d50c70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:24.653127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354b6c5-765f-480b-8ab5-fe12d50c70b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:24.653127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354b6c5-765f-480b-8ab5-fe12d50c70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:24.653314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354b6c5-765f-480b-8ab5-fe12d50c70b3 None admin] PATH_INFO: `/v3/system/users/eaa5ef83578845dabc6f22cccc1bc919/roles/790025671695475db6aa2a39c8746e54` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:24.654650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354b6c5-765f-480b-8ab5-fe12d50c70b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=eaa5ef83578845dabc6f22cccc1bc919, role_id=790025671695475db6aa2a39c8746e54)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:24.656567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354b6c5-765f-480b-8ab5-fe12d50c70b3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:24.763123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 170/334] 10.222.0.78 () {58 vars in 1334 bytes} [Wed Aug 10 22:52:24 2022] PUT /identity/v3/system/users/eaa5ef83578845dabc6f22cccc1bc919/roles/790025671695475db6aa2a39c8746e54 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:26.516475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271d50d-1775-4913-ac65-645ef23de7fc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:26.517019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271d50d-1775-4913-ac65-645ef23de7fc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:26.517019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271d50d-1775-4913-ac65-645ef23de7fc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:26.518369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 165/335] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:26.523951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:26.524803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:26.525260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:26.554133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:26.555367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:26.637511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 171/336] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:26.662310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271d50d-1775-4913-ac65-645ef23de7fc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:26.778499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271d50d-1775-4913-ac65-645ef23de7fc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:26.827349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:26.828555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:26.828837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:26.829107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:26.841708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:26.847092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:26.847971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:26.848785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 166/337] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:26.855709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:26.883055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:26.970249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1d309-d42a-4795-98e0-bf42c875b7cb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:27.010151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:27.010857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.011138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.011410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.012077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:27.012376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:27.013776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daece48-ba4b-41ba-bc89-a111ade1ee58 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:27.026353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 172/338] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:27.032348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:27.097591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:27.162925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98ea71-b7c4-4b4f-9a91-13e903fc3e9b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:27.197119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:27.198151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.198625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.199125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.268384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6240-7b16-46b8-bab6-6e5d50f850b7 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.268950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6240-7b16-46b8-bab6-6e5d50f850b7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.269433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6240-7b16-46b8-bab6-6e5d50f850b7 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.271005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 173/339] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:27.278581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:27.280145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:27.281067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.281519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.281938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.318116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:27.319654 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:27.379022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:27.470906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 167/340] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:27.478426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:27.488885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 174/341] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:27.547746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:27.560283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:27.626328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f604bc-0d44-4ec6-9064-a2d0aa252927 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:27.676799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b3938-c460-45a6-9fba-5070b21aaaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:27.677594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b3938-c460-45a6-9fba-5070b21aaaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.677878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b3938-c460-45a6-9fba-5070b21aaaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.678199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b3938-c460-45a6-9fba-5070b21aaaf0 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.680393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b3938-c460-45a6-9fba-5070b21aaaf0 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:27.682253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b3938-c460-45a6-9fba-5070b21aaaf0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:27.684297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 168/342] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:27 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:27.686388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358ac5e-0170-4070-897b-8067794b4dff None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.686388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358ac5e-0170-4070-897b-8067794b4dff None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.686582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358ac5e-0170-4070-897b-8067794b4dff None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.690913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 169/343] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:27.694602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358ac5e-0170-4070-897b-8067794b4dff None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:27.714604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c0c6-f9f8-407b-9767-796a86a31f4d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:27.747897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:27.778310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282eb97-2301-41af-ac82-d06171b5c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:27.778957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eb97-2301-41af-ac82-d06171b5c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.779230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eb97-2301-41af-ac82-d06171b5c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.779495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eb97-2301-41af-ac82-d06171b5c3f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.800315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282eb97-2301-41af-ac82-d06171b5c3f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a282eb97-2301-41af-ac82-d06171b5c3f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:27.806770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:27.807778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:27.808625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:27.811451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 175/344] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:27.813970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.814572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.815000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.849781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:27.850992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:27.886594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358ac5e-0170-4070-897b-8067794b4dff None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:27.943241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:27.944266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:27.944818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:27.945284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:27.952843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:27.954710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:27.957341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 170/345] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:27 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:27.958786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.002277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.012867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 176/346] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:28.017068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.085446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.105110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c42f1-a161-42ee-9128-d8c60c9111ff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.148168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.149002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.149441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.149857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.150696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:28.151003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:28.152075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2415b90-c60c-43e0-baa7-308065151a59 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.152377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347ebeb-bd9a-42ef-bcfd-49ffa71d4044 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:28.165180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 171/347] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:28.167446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795ac26-e051-4f63-89b0-17bdd9b35b8e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.171689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795ac26-e051-4f63-89b0-17bdd9b35b8e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.171974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795ac26-e051-4f63-89b0-17bdd9b35b8e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.173393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 172/348] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:28.174295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795ac26-e051-4f63-89b0-17bdd9b35b8e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.203363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.204430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.204710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.205139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.205863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=507e4cf5a05041debeb551f9c60aa218) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:28.206167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:28.207626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf89c18-1140-42d4-b8d5-4cd2b7a9894e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:28.213814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 177/349] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:28.215277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46074ffe-b3ad-46dd-8421-23dd31db66ce None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.215530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46074ffe-b3ad-46dd-8421-23dd31db66ce None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.215789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46074ffe-b3ad-46dd-8421-23dd31db66ce None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.217177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 178/350] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:28.217714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46074ffe-b3ad-46dd-8421-23dd31db66ce None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.311905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795ac26-e051-4f63-89b0-17bdd9b35b8e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.313310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.366578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef96854-3bd7-4081-ab10-31990de3a959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.367577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96854-3bd7-4081-ab10-31990de3a959 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.367760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46074ffe-b3ad-46dd-8421-23dd31db66ce None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.368041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96854-3bd7-4081-ab10-31990de3a959 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.368399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96854-3bd7-4081-ab10-31990de3a959 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.384632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef96854-3bd7-4081-ab10-31990de3a959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2ef96854-3bd7-4081-ab10-31990de3a959 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:28.392075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:28.393045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:52:28.393890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:28.400637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 173/351] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:28.400637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3d981-fad5-4d49-82e3-f71e843e8712 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.400637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3d981-fad5-4d49-82e3-f71e843e8712 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.400637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3d981-fad5-4d49-82e3-f71e843e8712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.420547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-34f3d981-fad5-4d49-82e3-f71e843e8712 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:28.421868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-34f3d981-fad5-4d49-82e3-f71e843e8712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:28.424854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.431231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.432188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.435268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.435268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.513085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 174/352] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:28.513362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:28.515086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:28.519312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.519312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.519576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.552076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:28.552822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:28.585235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.623817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 179/353] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:28.624734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.636650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 175/354] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:28.639281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.711895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.734452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d203205-cf41-418b-a6c4-17a0b584c634 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.747456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db213148-7bad-46a6-ac3b-a733fb20328b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.774797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.775929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.776210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.776485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.777160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:28.777456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:28.778888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:28.789247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 180/355] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:28.790393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.801420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.802070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.802340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.802604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.818036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:28.822892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:28.824552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:28.825402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:28.829126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 176/356] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:28.830037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:28.833799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:28.941202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c283c-ae20-452e-9d9f-888b03afe51b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.966996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee6b89-e583-4ea0-8d04-1108332b4cd9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:28.984818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438dc6a-d9be-4072-9125-33294d0f94ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:28.985536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438dc6a-d9be-4072-9125-33294d0f94ce None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.985823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438dc6a-d9be-4072-9125-33294d0f94ce None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.986230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438dc6a-d9be-4072-9125-33294d0f94ce None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.987394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438dc6a-d9be-4072-9125-33294d0f94ce None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:28.989144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438dc6a-d9be-4072-9125-33294d0f94ce None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:28.991047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 181/357] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:28.992482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831160f9-3a7b-4fd2-9df7-5f93229b69ca None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:28.992765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831160f9-3a7b-4fd2-9df7-5f93229b69ca None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:28.993037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831160f9-3a7b-4fd2-9df7-5f93229b69ca None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:28.994249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 182/358] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:28.994971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831160f9-3a7b-4fd2-9df7-5f93229b69ca None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.010364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.019710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.020160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.020435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.020708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.037872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:29.042863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:29.043834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:29.044630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:29.045239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 177/359] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:29.045845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.133446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.142740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831160f9-3a7b-4fd2-9df7-5f93229b69ca None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.185049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb0bb-8a75-4965-8569-0295cbafed0b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.196859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ee705-9cfa-4c12-9f99-155427908b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.197818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee705-9cfa-4c12-9f99-155427908b99 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.198884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee705-9cfa-4c12-9f99-155427908b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.199339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee705-9cfa-4c12-9f99-155427908b99 None admin] PATH_INFO: `/v3/users/eaa5ef83578845dabc6f22cccc1bc919` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.201152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ee705-9cfa-4c12-9f99-155427908b99 None admin] RBAC: Authorizing `identity:get_user(user_id=eaa5ef83578845dabc6f22cccc1bc919)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:29.204221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ee705-9cfa-4c12-9f99-155427908b99 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:29.208040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 183/360] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:52:28 2022] GET /identity/v3/users/eaa5ef83578845dabc6f22cccc1bc919 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:29.210131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.210567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.211064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.238902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:29.239431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.240037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:29.240144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.240301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.240626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.241338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:29.241637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:29.243618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:29.252500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 178/361] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:29.253182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.317748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.337321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 184/362] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:29.338190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.377559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34a3d7-2830-4495-9045-b2fe79265d62 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.425885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.426912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.427190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.427457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.429844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:29.431314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:29.433914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 179/363] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:29.433914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.493995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.504581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6db088-87b4-4c0f-8f2f-e604970a7ced None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.542862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.543762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.545131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.545547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.546378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:29.546987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:29.548363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:29.558856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 185/364] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:29.558856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.579587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e813-9660-4298-aa6d-a4bc032b1dd2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.600372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.621341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.621959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.622236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.622504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.623260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] RBAC: Adding query filter params (role.id=790025671695475db6aa2a39c8746e54, user.id=eaa5ef83578845dabc6f22cccc1bc919, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:29.623614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:29.625629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:29.640326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 180/365] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/role_assignments?user.id=eaa5ef83578845dabc6f22cccc1bc919&scope.system=all&role.id=790025671695475db6aa2a39c8746e54 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:29.642321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.698188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc74ed-cf6c-40a3-8b5f-2f9aa6c97f1a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.746010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.747646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.748147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.748632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.779381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.781519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3661730-cec2-47b1-a865-cfb518638d5f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:29.832344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:29.833320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:29.833768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:29.834405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:29.846894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:29.848462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:29.848951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:29.854986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:29.856858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:29.857916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 181/366] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:29.858589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.907855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:29.983598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 186/367] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:29.985201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:29.991410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7c31-0ad4-41d7-a025-26e8b52f8cac None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.036224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.036733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.038047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.038047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.038047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=507e4cf5a05041debeb551f9c60aa218) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:30.038278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.039731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.051328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 182/368] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:30.051328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.052003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.120064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61ed3-db6f-4f8d-8323-5bbf394af4f3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.162441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.163538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.163538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.163734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.175050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.179301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4927f-3346-4307-88a5-c92dfabb101d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.228760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.229883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.230313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.230977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.232234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:30.232688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.234791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.245665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 183/369] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:30.246928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.257728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.258985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.320258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.354955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10510f7-da65-4033-b013-38700ad82cbe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.378694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 187/370] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:30.379436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.392894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.393870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.394383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.394898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.396343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.398074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.400346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 184/371] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:30.401327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.450998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.540710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0f3aa-fe94-4f53-b005-dc836e12bfd1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.543991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5311d5-8208-4323-8de8-934003d377c4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.582290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.584668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.584780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.584889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.584889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.585015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.585015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.585015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.585445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.586921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.588734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 185/372] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:30.589337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.611671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.681801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.683900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.745425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.753735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5f1d-30bd-4fa2-add4-39beda3fbcf0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.797148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ffeb1-494a-4bac-8d9e-a609abfd9851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.798123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ffeb1-494a-4bac-8d9e-a609abfd9851 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.798552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ffeb1-494a-4bac-8d9e-a609abfd9851 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.799107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ffeb1-494a-4bac-8d9e-a609abfd9851 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.801189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ffeb1-494a-4bac-8d9e-a609abfd9851 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.803140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ffeb1-494a-4bac-8d9e-a609abfd9851 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.805611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 186/373] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:30.806963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a03d-c457-4b2c-813e-ac2e1cff3a45 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.807392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a03d-c457-4b2c-813e-ac2e1cff3a45 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.807815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a03d-c457-4b2c-813e-ac2e1cff3a45 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.809005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 187/374] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:30.809820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a03d-c457-4b2c-813e-ac2e1cff3a45 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.839344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 188/375] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:30.840721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.918097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:30.970934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a03d-c457-4b2c-813e-ac2e1cff3a45 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.994838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5142e0-8515-4963-bfb9-01c98fa4745f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.013099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46b0bc-0203-4080-a649-6daf3e6787f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.014097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46b0bc-0203-4080-a649-6daf3e6787f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.014545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46b0bc-0203-4080-a649-6daf3e6787f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.015053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46b0bc-0203-4080-a649-6daf3e6787f3 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.016350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46b0bc-0203-4080-a649-6daf3e6787f3 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.018175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46b0bc-0203-4080-a649-6daf3e6787f3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.019968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 188/376] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:31.021257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.021709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.024831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.040595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.041199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.041469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.041752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.042462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=6e772a239a88438d8a76221f9b346b3a) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:31.042873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.044319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.046179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:31.048284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:31.050510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 189/377] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:31.051341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.082416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:31.135102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 189/378] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:31.136277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.196498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232c4b5-f88a-4d80-8ae6-e8843d171480 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.225221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:31.251353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.252171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.253836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.254209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] PATH_INFO: `/v3/projects/669ea2e08a3a4675928c19c0f5a75735` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.255608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] RBAC: Authorizing `identity:get_project(project_id=669ea2e08a3a4675928c19c0f5a75735)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.257154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.259101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 190/379] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/projects/669ea2e08a3a4675928c19c0f5a75735 => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:31.259991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.289166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c961d-cd12-47c2-9211-7d52f54a7429 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.342990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.344503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.345208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.345655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.347491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=6e772a239a88438d8a76221f9b346b3a) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:31.348735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.351010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.359818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 190/380] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:31.361295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.409733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:31.454021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a4c2d-805e-4820-9b28-8161bf99d017 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.528339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.528979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.529254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.529524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.551036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:31.556905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:31.557815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:31.558628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 191/381] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:31.565271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.566106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db2107-2741-40f4-8b09-b549b05cd740 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.640716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:31.641030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.641842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.642366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.643298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] PATH_INFO: `/v3/groups/4beac314b9c74b508f577bcce0888250` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.650096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] RBAC: Authorizing `identity:get_group(group_id=4beac314b9c74b508f577bcce0888250)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.651806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.653853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 191/382] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/groups/4beac314b9c74b508f577bcce0888250 => generated 229 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:31.660888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.717751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2df19-d8fa-4ac7-b9b3-b711fd4c21fd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.760522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.761373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.761796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.762210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.763085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:31.764089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.766166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.775606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 192/383] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:31.783138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:31.807213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:31.820494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7b5af-3426-430d-a609-1c2a669d0106 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.891710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:31.892571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:31.892992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:31.893404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:31.894298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] RBAC: Adding query filter params (group.id=4beac314b9c74b508f577bcce0888250, role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=669ea2e08a3a4675928c19c0f5a75735) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:31.894781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:31.896357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:31.906990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 192/384] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/role_assignments?group.id=4beac314b9c74b508f577bcce0888250&scope.project.id=669ea2e08a3a4675928c19c0f5a75735&role.id=be98a6e301f144d88c992f65bca561fb => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:31.979625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97780564-3e68-4587-ab30-a0f8d19979b1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:31.989573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:32.028997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:32.029618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:32.029891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:32.030157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:32.035643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:32.037153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:32.039233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 193/385] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:31 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:32.044790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:32.193927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5b11e-1633-4189-b57d-db8670c865dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:32.244115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0efd7f-6581-40a5-902a-a3482abd38ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:32.244674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0efd7f-6581-40a5-902a-a3482abd38ce None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:32.244951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0efd7f-6581-40a5-902a-a3482abd38ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:32.245266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0efd7f-6581-40a5-902a-a3482abd38ce None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:32.246380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0efd7f-6581-40a5-902a-a3482abd38ce None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:32.247854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0efd7f-6581-40a5-902a-a3482abd38ce None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:32.251257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 193/386] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:32 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:32.255870 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:32.395248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d9f4-73e2-4a53-bd3d-ca446a0cfea8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:32.445411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:32.446139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:32.446422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:32.446728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218/groups/1524399e7c4441a6b971c28a7544eceb/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:32.448729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=507e4cf5a05041debeb551f9c60aa218, group_id=1524399e7c4441a6b971c28a7544eceb, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:32.450165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:32.509288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed739cc7-1cfa-49a7-ad4c-ef3ac50cd16b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:32.509868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed739cc7-1cfa-49a7-ad4c-ef3ac50cd16b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:32.510336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed739cc7-1cfa-49a7-ad4c-ef3ac50cd16b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:32.512175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 194/387] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:32.517329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:32.517842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:32.522833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:32.537093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:32.538571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:32.557521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:32.588196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 194/388] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 10 22:52:32 2022] PUT /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218/groups/1524399e7c4441a6b971c28a7544eceb/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:32.616641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 195/389] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:32.659887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:32.714122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:32.807871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee768c0-0953-4c34-9d86-cfae06bf48a9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:32.850134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:32.850806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:32.851082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:32.851348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:32.872177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:32.880359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:32.881376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:32.882180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:32.885994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 195/390] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:32.887245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.019296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47d423-31aa-431a-a19a-32f24395f6b0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:33.069448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:33.069978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.070249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.070516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.071256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:33.071554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:33.073135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:33.082309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 196/391] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:33.087536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.196066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c5c13-9a09-4c05-939a-ebe4ff7a79e7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:33.197626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:33.230810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c18e6-55fd-4073-97e2-f0c72ba5de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:33.231436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c18e6-55fd-4073-97e2-f0c72ba5de4c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.231742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c18e6-55fd-4073-97e2-f0c72ba5de4c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.232019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c18e6-55fd-4073-97e2-f0c72ba5de4c None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.233119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c18e6-55fd-4073-97e2-f0c72ba5de4c None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:33.235093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c18e6-55fd-4073-97e2-f0c72ba5de4c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:33.236994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 196/392] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:33 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:33.247778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.270262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:33.377642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73308728-aaed-49fe-9cf2-b123c071bc93 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:33.419620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:33.425653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.425926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.426203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.427489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:33.428940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:33.430948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 197/393] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:33 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:33.436441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.436889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87bc2-36ef-4974-b27c-384c0049fef8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.437902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87bc2-36ef-4974-b27c-384c0049fef8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.438409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87bc2-36ef-4974-b27c-384c0049fef8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.439942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 197/394] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:33.445153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.445657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.446118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.470020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:33.471746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:33.566563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:33.584779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 198/395] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:33.587627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189af15-68a5-4dfd-a56c-928a0275573b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:33.620217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.645227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:33.645819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.646098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.646372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218/groups/1524399e7c4441a6b971c28a7544eceb/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.652232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] RBAC: Authorizing `identity:create_grant(project_id=507e4cf5a05041debeb551f9c60aa218, group_id=1524399e7c4441a6b971c28a7544eceb, role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:33.653692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:33.728001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:33.737506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d4e1e-de82-431f-b0ca-d8283b7a1361 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:33.751678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 198/396] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 10 22:52:33 2022] PUT /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218/groups/1524399e7c4441a6b971c28a7544eceb/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:52:33.795383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:33.796078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.796377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.796686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.817565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:33.822473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:33.823438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:33.824236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:33.829479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 199/397] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:33.830944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.980388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3a9a4-52e3-407e-811e-422b397ecb08 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:34.017423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:34.018028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.018303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.018573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.019262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:34.019559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:34.020916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38dbf5-6c07-41bc-967e-f6294e184c23 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:34.034699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 199/398] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:34.037010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.146182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457506d-d21b-4a48-82de-7cf67db93851 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:34.179641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af184913-c829-46a8-8763-7571579fc379 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.179965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af184913-c829-46a8-8763-7571579fc379 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.180171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af184913-c829-46a8-8763-7571579fc379 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.181770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 200/399] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:34.186370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.186646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.186975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.193678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:34.194308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.194580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.194868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.196840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:34.202089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:34.204273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 200/400] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:34 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:34.211307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:34.214800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.223456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:34.224652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:34.261663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:34.355758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe076eb6-b0f1-44b3-9067-f747e588db2c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:34.376625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 201/401] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:34.390983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:34.391599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.391874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.392140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.393294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:34.394758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:34.396705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 201/402] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:34 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:34.402134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.409595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.518305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:34.530361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe5a79-48df-4869-81ca-bed2c25f7f69 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:34.534723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d11e-4ffb-48ac-a529-023930b54e04 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:34.569435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047334b1-053d-48f2-9f6b-5952d13ed9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:34.569865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047334b1-053d-48f2-9f6b-5952d13ed9b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.570198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047334b1-053d-48f2-9f6b-5952d13ed9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.570472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047334b1-053d-48f2-9f6b-5952d13ed9b7 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a/groups/1524399e7c4441a6b971c28a7544eceb/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.572506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047334b1-053d-48f2-9f6b-5952d13ed9b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e772a239a88438d8a76221f9b346b3a, group_id=1524399e7c4441a6b971c28a7544eceb, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:34.574009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047334b1-053d-48f2-9f6b-5952d13ed9b7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:34.578104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:34.582507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c152-1a04-4f70-8894-8d8b9208b495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:34.583648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c152-1a04-4f70-8894-8d8b9208b495 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.583984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c152-1a04-4f70-8894-8d8b9208b495 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.595625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c152-1a04-4f70-8894-8d8b9208b495 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.595751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c152-1a04-4f70-8894-8d8b9208b495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5a90c152-1a04-4f70-8894-8d8b9208b495 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:34.600392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:34.601257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:34.602128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 202/403] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:34.603405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb119f-de4b-409e-a36d-14f36113d6d3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.603405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb119f-de4b-409e-a36d-14f36113d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.603636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb119f-de4b-409e-a36d-14f36113d6d3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.606712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 203/404] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:34.607145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb119f-de4b-409e-a36d-14f36113d6d3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.691109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 202/405] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 10 22:52:34 2022] PUT /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a/groups/1524399e7c4441a6b971c28a7544eceb/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:34.692742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.693347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.693810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.721039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:34.725945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:34.765533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb119f-de4b-409e-a36d-14f36113d6d3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:34.841243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 203/406] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:34.860042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:34.860911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:34.861408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:34.861834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:34.862675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:34.863077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.863278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:34.864883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:34.876355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 204/407] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:34.882100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:34.899038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:34.991386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e167bb2-5579-4998-9b9c-336d0d547c20 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.001633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764b6e-cfa7-4729-88a0-fd496acfcbff None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.019490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:35.029035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.029485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.029760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.030022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.042761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.043964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.044071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:35.044538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.045009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.046438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:35.048123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:35.048878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:35.049771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:35.050567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 204/408] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:35.052149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 205/409] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:34 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:35.056427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.057984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.184003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229a93-bccb-415d-83ea-cdbbab52bd9b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.224344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:35.225598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.226437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.227412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.228335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:35.228894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:35.230807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7ed8-3466-487b-a272-bd03a0b9aee1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:35.243844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 205/410] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:35.245536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a366bf8-d87b-4014-9d14-045b69d3a0d2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.246051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a366bf8-d87b-4014-9d14-045b69d3a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.246589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a366bf8-d87b-4014-9d14-045b69d3a0d2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.248733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 206/411] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:35.250419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a366bf8-d87b-4014-9d14-045b69d3a0d2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.261038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484099b-d9ef-46cd-a6f0-309148ba6abd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.299456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5542cf2-abdc-4945-8367-6113363e70cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.300093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5542cf2-abdc-4945-8367-6113363e70cd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.300374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5542cf2-abdc-4945-8367-6113363e70cd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.300650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5542cf2-abdc-4945-8367-6113363e70cd None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.301784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5542cf2-abdc-4945-8367-6113363e70cd None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:35.303281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5542cf2-abdc-4945-8367-6113363e70cd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:35.305198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 206/412] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:35 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:35.310869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.310869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.310869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.322378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:35.334029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:35.336073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:35.384100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a366bf8-d87b-4014-9d14-045b69d3a0d2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.438751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 207/413] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:35.438751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.450447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.451120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.451402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.451676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] PATH_INFO: `/v3/groups/4beac314b9c74b508f577bcce0888250` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.454720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] RBAC: Authorizing `identity:get_group(group_id=4beac314b9c74b508f577bcce0888250)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:35.456165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:35.457902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 207/414] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:35 2022] GET /identity/v3/groups/4beac314b9c74b508f577bcce0888250 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:35.463823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.531065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:35.555658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0ea0b-1365-45a4-82f6-69035f6cb8a4 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.570949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3383494-7706-4441-8762-453ce4e3df07 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.588052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:35.593401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.594244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.594737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.601183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a/groups/1524399e7c4441a6b971c28a7544eceb/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.601183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e772a239a88438d8a76221f9b346b3a, group_id=1524399e7c4441a6b971c28a7544eceb, role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:35.601183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:35.607537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.608170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.608450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.609264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] PATH_INFO: `/v3/projects/669ea2e08a3a4675928c19c0f5a75735` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.610387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] RBAC: Authorizing `identity:get_project(project_id=669ea2e08a3a4675928c19c0f5a75735)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:35.611907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:35.617830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 208/415] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:35 2022] GET /identity/v3/projects/669ea2e08a3a4675928c19c0f5a75735 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:35.617830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.720991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77212a-4397-4ff0-8166-018835ce0f5b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.755333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:35.755960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:35.756244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:35.756513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:35.774689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:35.775580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:35.776444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 209/417] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:35.777206 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.777983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 208/417] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 10 22:52:35 2022] PUT /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a/groups/1524399e7c4441a6b971c28a7544eceb/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:35.797312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:35.885308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:35.986877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c7cbc-c54d-4800-b9e4-f1a7cba98674 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:35.987793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9ebd-4984-4a12-8aee-5824f1215a1d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:36.026940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:36.027144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8831c-a89e-4903-8319-f2b09b716d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:36.027702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8831c-a89e-4903-8319-f2b09b716d5c None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.027985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8831c-a89e-4903-8319-f2b09b716d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.028258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8831c-a89e-4903-8319-f2b09b716d5c None admin] PATH_INFO: `/v3/projects/669ea2e08a3a4675928c19c0f5a75735/groups/4beac314b9c74b508f577bcce0888250/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.028927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.029410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.029869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.030220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8831c-a89e-4903-8319-f2b09b716d5c None admin] RBAC: Authorizing `identity:create_grant(project_id=669ea2e08a3a4675928c19c0f5a75735, group_id=4beac314b9c74b508f577bcce0888250, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:36.031570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:36.032067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:36.032251 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8831c-a89e-4903-8319-f2b09b716d5c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:36.033783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:36.040223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:36.044367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 209/418] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:36.050399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:36.147494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 210/419] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 10 22:52:35 2022] PUT /identity/v3/projects/669ea2e08a3a4675928c19c0f5a75735/groups/4beac314b9c74b508f577bcce0888250/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:36.156490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13051-eaf6-47e6-bd75-a2871b2d7632 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.156782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13051-eaf6-47e6-bd75-a2871b2d7632 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.157192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13051-eaf6-47e6-bd75-a2871b2d7632 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.159802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 211/420] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:36.165294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.165658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.166108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.170448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6413-5b0c-42a0-865f-2eecb0cbfdee None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:36.208230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:36.211001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:36.235460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:36.237778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.237993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.238265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.351359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 212/421] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:36.364124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:36.365770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:36.367306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:36.384643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:36.449067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:36.508315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 210/422] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:36.514338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:36.593655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6fbc5-426f-4c74-9d2e-217dbf5989f3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:36.630896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:36.635840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d66f6-1a74-47f4-8fb7-b1b34385cc46 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:36.676767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:36.677658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.678529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.678997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.681342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:36.683244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:36.685064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 211/423] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:36 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:36.691190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:36.692624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:36.693346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.693715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.694056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.721607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:36.726565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:36.727482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:36.728314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 213/424] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:36.751246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:36.814876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060cb8-003b-4cae-8829-429550c0f103 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:36.854293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:36.855213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:36.855632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:36.856040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:36.857354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:36.859615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:36.862324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 212/425] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:36 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:36.868894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:36.909609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:36.993209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fd60f-fa66-4889-ba66-95978478eb2f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.012327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.034885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f024d-cf92-4a6e-ae7a-308f12dc7b62 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.051442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.052101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.052326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.052591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.053321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=507e4cf5a05041debeb551f9c60aa218) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:37.053629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.055218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4266ca-09aa-4186-b94a-7b0f9721a9d0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:37.073271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 213/426] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:36 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:37.073878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.074554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14874839-0b2d-48ed-b724-5cb70ff59f2f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.074864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14874839-0b2d-48ed-b724-5cb70ff59f2f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.075086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.075196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14874839-0b2d-48ed-b724-5cb70ff59f2f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.075593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.076025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.076580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 214/427] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:37.077031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:37.077860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.080976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:37.082629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.083097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.083502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.098685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 214/428] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:37.098685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.135893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:37.137382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:37.214130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfde40-983d-4323-98a8-40220f22a7ee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.254776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.255383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.255666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.255935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.290619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.292932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 215/429] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:37.319798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.353357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.354731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.354924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:37.455221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af6e2a-65cf-4531-bd76-01d2f5aeb3ba None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.493605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 215/430] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:37.498375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.498375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.498375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.498375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.500940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.511038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:37.516673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:37.517570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:37.518375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 216/431] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:37.523584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.572064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.626536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bd45c-bd30-4661-b290-cb5da0aedee5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.647024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.661041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e553aa-09b2-434c-a6fa-414a403a6785 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.672982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.673621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.673899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.674170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] PATH_INFO: `/v3/projects/507e4cf5a05041debeb551f9c60aa218` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.675363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] RBAC: Authorizing `identity:get_project(project_id=507e4cf5a05041debeb551f9c60aa218)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.676818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:37.678809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 216/432] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/projects/507e4cf5a05041debeb551f9c60aa218 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:37.688258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.710213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.710422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.710766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.710988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.711651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:37.711942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.713309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:37.722450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 217/433] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:37.727615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.810619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dccbfb-907e-475a-90ff-cc036ddc80e2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.842197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d4ced-6f51-4dfc-b27e-dcb46c1374a9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:37.843567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.852321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.853232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.859569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.859569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.859569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.859569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:37.860172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 217/434] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:37.866108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:37.885225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:37.885752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:37.886029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:37.886299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:37.922212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:37.984807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:37.986509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.000839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98145dc9-688d-4de5-9db6-c1751b4df675 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.047383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.048991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.049434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.049868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.050843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=507e4cf5a05041debeb551f9c60aa218) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:38.051418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:38.053162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6c8c2-79c2-49bd-8afb-0867cc27db45 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.070872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 218/435] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=507e4cf5a05041debeb551f9c60aa218&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:38.083455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de77ff-624f-4f3e-aacb-5548815ed04b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.083755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de77ff-624f-4f3e-aacb-5548815ed04b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.084013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de77ff-624f-4f3e-aacb-5548815ed04b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.085672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 219/436] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:38.091371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.091662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.091919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.112080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:38.112647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 218/437] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:38.113298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:38.118046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.177377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:38.198109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:38.203133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 220/438] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:38.219732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.264766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3b57a-a10e-4908-b5ef-b5f596836a13 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.314641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.315426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.315653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.315928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.317786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:38.319788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.322064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 219/439] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:38.327917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.353136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29031ed6-7e30-4afd-a58f-385468d309dc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.400550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.401191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.401467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.401730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.419414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:38.428741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:38.429833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:38.430855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 221/440] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:38.439211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.471284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3c96c-c65c-4905-be48-81d0729aa7f1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.491098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:38.506209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:38.516144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.518313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.518313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.518313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.519645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:38.521095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.522853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 220/441] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:38.528364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.563505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d1c24-a59b-4bee-b582-13d73cc63dda None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.601164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.601767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.602036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.602304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.602991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:38.603288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:38.604668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.614148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 222/442] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:38.619844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.636318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1660c46-8218-43d4-8cdf-6748defba6c2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.672866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.673468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.673747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.674019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.674775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=f8c74cd9c7d245249553a39801f08d82, scope.project.id=6e772a239a88438d8a76221f9b346b3a) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:38.675084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:38.676590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80995b77-b4ad-40cb-84c4-08001e4dabce None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.682755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 221/443] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=f8c74cd9c7d245249553a39801f08d82 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:38.683883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e847c-2b73-4c97-935b-13c9cc70c22e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.684167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e847c-2b73-4c97-935b-13c9cc70c22e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.684438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e847c-2b73-4c97-935b-13c9cc70c22e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.687218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 222/444] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:38.699832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.701619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.701619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.720435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:38.721730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:38.757511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062aa6c-9f4e-4c14-b6a7-a7c226b62866 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:38.769784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:38.778919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:38.801754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:38.802374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:38.802684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:38.803020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:38.804740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 223/445] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:38.827986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:38.894394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:38.896616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:38.950594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f5709-5f3d-41dd-80db-140deb38b77b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.008301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.008301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.008591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.008791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.027311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:39.032648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:39.035884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:39.036706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 224/446] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:39.037373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 223/447] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:39.041081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.041274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.052785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.096302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.155650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3947c0-00b2-4a73-a685-d7d89c95a1a9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.159009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206507c-16e9-4495-b8ce-2e09d44d2d0d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.193625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.194252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.194532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.194838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] PATH_INFO: `/v3/projects/6e772a239a88438d8a76221f9b346b3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.195965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.196121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] RBAC: Authorizing `identity:get_project(project_id=6e772a239a88438d8a76221f9b346b3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.196563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.196874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.197107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.197421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.197768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:39.198068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.199094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 224/448] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/projects/6e772a239a88438d8a76221f9b346b3a => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:39.199481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.204804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.211920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 225/449] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:39.217477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.287346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.310367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea9ad5-c991-48cf-8bf4-9cf178680001 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.322961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3d23e-aaec-4668-ae34-92a11f46f8cb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.324823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.343750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.344377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.344649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.344918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] PATH_INFO: `/v3/groups/1524399e7c4441a6b971c28a7544eceb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.346028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] RBAC: Authorizing `identity:get_group(group_id=1524399e7c4441a6b971c28a7544eceb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.347471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.349023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 225/450] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/groups/1524399e7c4441a6b971c28a7544eceb => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:39.354900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.356130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.356765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.357043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.357482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.431877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.433729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.455668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc07508-8dee-4367-9c43-df61281847c0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.495565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.496453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.496718 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.497428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] RBAC: Adding query filter params (group.id=1524399e7c4441a6b971c28a7544eceb, role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=6e772a239a88438d8a76221f9b346b3a) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:39.497727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.499738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.505523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 226/451] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/role_assignments?group.id=1524399e7c4441a6b971c28a7544eceb&scope.project.id=6e772a239a88438d8a76221f9b346b3a&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:39.540659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 226/452] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:52:39.541910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.554117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.646911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19677f-0c0a-4e68-b1b9-7b460bc19105 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.680271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.680893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.681172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.681439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] PATH_INFO: `/v3/projects/669ea2e08a3a4675928c19c0f5a75735` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.682742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] RBAC: Authorizing `identity:get_project(project_id=669ea2e08a3a4675928c19c0f5a75735)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.684249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.685848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 227/453] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/projects/669ea2e08a3a4675928c19c0f5a75735 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:39.692016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.694174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.792858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725d851-c1f5-4e07-811d-88189a51c65e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.826767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43525590-46ab-41ef-9625-a845dd630a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.827972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525590-46ab-41ef-9625-a845dd630a31 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.828239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525590-46ab-41ef-9625-a845dd630a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.828500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525590-46ab-41ef-9625-a845dd630a31 None admin] PATH_INFO: `/v3/groups/4beac314b9c74b508f577bcce0888250` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.829585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43525590-46ab-41ef-9625-a845dd630a31 None admin] RBAC: Authorizing `identity:get_group(group_id=4beac314b9c74b508f577bcce0888250)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.831071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43525590-46ab-41ef-9625-a845dd630a31 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.832606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 227/454] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/groups/4beac314b9c74b508f577bcce0888250 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:39.838425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:39.914012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:39.936219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d4d78-8354-4a12-84e7-f569d08cbce7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:39.968722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:39.969311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:39.969587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:39.969856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:39.970562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] RBAC: Adding query filter params (group.id=4beac314b9c74b508f577bcce0888250, role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=669ea2e08a3a4675928c19c0f5a75735) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:39.970900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:39.972337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ed8e-5c63-4c4f-9761-2d2351a6ecae None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:39.982252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 228/455] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 10 22:52:39 2022] GET /identity/v3/role_assignments?group.id=4beac314b9c74b508f577bcce0888250&scope.project.id=669ea2e08a3a4675928c19c0f5a75735&role.id=be98a6e301f144d88c992f65bca561fb => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:42.344052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7d94-d98e-4955-ae8d-d10b91c14750 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.345066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7d94-d98e-4955-ae8d-d10b91c14750 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.347128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7d94-d98e-4955-ae8d-d10b91c14750 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.349056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 228/456] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:42.356183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c922-e3d6-43c8-9a37-fd1484661564 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.356669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c922-e3d6-43c8-9a37-fd1484661564 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.356743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c922-e3d6-43c8-9a37-fd1484661564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.386508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0ff6c922-e3d6-43c8-9a37-fd1484661564 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:42.387746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ff6c922-e3d6-43c8-9a37-fd1484661564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:42.488531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 229/457] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:42.540491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7d94-d98e-4955-ae8d-d10b91c14750 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:42.637202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ded3d-1a7e-45eb-9510-417d2b217729 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.637723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ded3d-1a7e-45eb-9510-417d2b217729 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.638165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ded3d-1a7e-45eb-9510-417d2b217729 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.643202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 230/458] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:42.648292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:42.651637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0c85d-f6f7-44cb-8403-5d69aee40948 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.652165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0c85d-f6f7-44cb-8403-5d69aee40948 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.652636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0c85d-f6f7-44cb-8403-5d69aee40948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.682618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-64a0c85d-f6f7-44cb-8403-5d69aee40948 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:42.684216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-64a0c85d-f6f7-44cb-8403-5d69aee40948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:42.693418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7d94-d98e-4955-ae8d-d10b91c14750 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:42.729169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:42.730097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.730582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.731075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.741736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:42.743443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:42.754366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:42.759517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 231/459] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:42.760968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f365c-ce33-47df-b593-cb58534d37c3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.761379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f365c-ce33-47df-b593-cb58534d37c3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.761776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f365c-ce33-47df-b593-cb58534d37c3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.766799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 232/460] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:42.767922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:42.772888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:42.773766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:42.773766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:42.773766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:42.773931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.774001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 229/461] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:42.774075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.774075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.778143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:42.788961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:42.790391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:42.878849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 233/462] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:42.879989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:42.891217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56d574-7b32-4e10-befc-9b441d338329 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:42.914889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:42.927729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:42.928655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.929212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.929687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.930530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:42.931075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:42.932662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:42.950690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 230/463] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:42.952047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.009609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf9419-edaf-4a2c-9244-ed10168c953c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.047646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.048312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.048588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.048853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.059915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.060265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d63f04-433d-42ff-825a-bc91bbb9a1d0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.062013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.062641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:43.079274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:43.080245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:43.081142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:43.081142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:43.081142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:43.081142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 234/464] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:43.081142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.113198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.114371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.114835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.115517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.130278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.131953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:43.148981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:43.150130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:43.151250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:43.151250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:43.151250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:43.157291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 231/465] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:43.157864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.195372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.223213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9209f01-6a1a-4da7-a7cc-4b121fa28b9d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.260858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864d0a8-035e-42d2-af67-8b5f08f8cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.261579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864d0a8-035e-42d2-af67-8b5f08f8cfc8 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.261927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864d0a8-035e-42d2-af67-8b5f08f8cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.262268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864d0a8-035e-42d2-af67-8b5f08f8cfc8 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.263069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864d0a8-035e-42d2-af67-8b5f08f8cfc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.264589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864d0a8-035e-42d2-af67-8b5f08f8cfc8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.285383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36fdb7-677c-4b58-a4f1-ac6a2200eaa4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.319042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.319629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.319903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.320176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.320790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:43.322984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.322984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.333968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 232/466] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:43.335533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.426839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.435215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6928387-0fc1-4ace-939d-39a1f83d72b8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.444068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 235/467] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:43 2022] POST /identity/v3/users => generated 246 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:43.444068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe2e9-243a-4ba9-aabf-f6b14b948600 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.444068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe2e9-243a-4ba9-aabf-f6b14b948600 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.444068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe2e9-243a-4ba9-aabf-f6b14b948600 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.444068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 236/468] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:43.444068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe2e9-243a-4ba9-aabf-f6b14b948600 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.479871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.481045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.481504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.482032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.482910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:43.484074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.485655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcc81f-b437-4358-8b72-6f40c0e1c2e1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.493250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.501354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 233/469] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:43.503058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.503522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.503993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.525996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:43.527621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:43.577120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe2e9-243a-4ba9-aabf-f6b14b948600 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.606268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 234/470] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:43.607474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.623020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.623763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.624047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.624317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.630137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.630137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.704798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.722420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd3b0d-10da-4dde-87e4-f3a01aaa4888 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.756000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af892c87-deb4-429b-8e0d-d8aeb0e870cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.756613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af892c87-deb4-429b-8e0d-d8aeb0e870cf None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.756889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af892c87-deb4-429b-8e0d-d8aeb0e870cf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.757157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af892c87-deb4-429b-8e0d-d8aeb0e870cf None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.757787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af892c87-deb4-429b-8e0d-d8aeb0e870cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:43.759248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af892c87-deb4-429b-8e0d-d8aeb0e870cf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:43.822411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 237/471] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:43 2022] POST /identity/v3/users => generated 248 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:43.827490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.923628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 235/472] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:43 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:43.962406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.973176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d798e-ab64-41bc-8480-545624ebe8ca None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:44.033403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:44.034062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.034335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.034604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.048577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094c5e-7a4b-4159-af2d-6306396f05d5 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.048889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094c5e-7a4b-4159-af2d-6306396f05d5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.049144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094c5e-7a4b-4159-af2d-6306396f05d5 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.050487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 236/473] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:44.053847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:44.055633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:44.056987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.057277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.057555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:44.073830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:44.074875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:44.075742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:44.075742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:44.075742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:44.075742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 238/474] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:44.081711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:44.127893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:44.129099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:44.212086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e664522-48ed-4414-a035-aaa4bbe7b64e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:44.223381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 237/475] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:44.255574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:44.262748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:44.263667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.264885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.265478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.266850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:44.267395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:44.269799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:44.287784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 239/476] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:44.292464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:44.303195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:44.345489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:44.398174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd7086-3b07-4929-a059-d54e91e3d601 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:44.445698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25765f7-063c-4695-9ab4-668a7c1946c2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:44.488578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:44.489106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.489385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.489726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.490290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:44.491749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:44.504363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:44.504976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.505253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.505569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.506230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:44.507710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:44.572300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.sql.core [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 22:52:44.572300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 22:52:44.572300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [parameters: {'id': 'ed32e3450ec44d00893a5c10271cc682', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 22:52:44.572300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81639) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result.read() Aug 10 22:52:44.588140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:52:44.589125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application yield resource Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise self.value Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application session.commit() Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 22:52:44.590116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.session.flush() Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application compat.raise_( Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 22:52:44.591093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:52:44.593399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result.read() Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:52:44.594284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [parameters: {'id': 'ed32e3450ec44d00893a5c10271cc682', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:44.595187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 22:52:44.596033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:44.596915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 238/477] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 10 22:52:44 2022] POST /identity/v3/roles => generated 144 bytes in 335 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:44.596915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:44.690262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 240/478] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:44 2022] POST /identity/v3/users => generated 249 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:44.712083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68466ec-b602-4713-9080-a92552317cd1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:44.758368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c2939-f7f1-4641-be35-4e3727d97d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:44.758955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c2939-f7f1-4641-be35-4e3727d97d8e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.759242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c2939-f7f1-4641-be35-4e3727d97d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.759509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c2939-f7f1-4641-be35-4e3727d97d8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.761664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:44.777470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c2939-f7f1-4641-be35-4e3727d97d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2b4c2939-f7f1-4641-be35-4e3727d97d8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:44.782323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:44.783325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:52:44.784149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:44.788345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 239/479] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:52:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:44.790180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:44.895724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:44.939238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98f0b-e239-45c1-b8bd-958a497700f4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:44.976148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:44.977145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:44.977643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:44.978133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:44.979021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:44.979565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:44.981140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e5c1d-7a72-4a90-9062-7ef1fae379a5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:44.990554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 241/480] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:45.944647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95ff9-be23-44de-866d-8f5a23f85311 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:45.945747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95ff9-be23-44de-866d-8f5a23f85311 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:45.946236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95ff9-be23-44de-866d-8f5a23f85311 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:45.948181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 240/481] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:45.953711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61458f0f-947c-4882-8f09-f871570dbb3b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:45.954548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61458f0f-947c-4882-8f09-f871570dbb3b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:45.955055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61458f0f-947c-4882-8f09-f871570dbb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:45.982223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-61458f0f-947c-4882-8f09-f871570dbb3b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:45.983859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-61458f0f-947c-4882-8f09-f871570dbb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:46.069070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 242/482] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:46.106076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95ff9-be23-44de-866d-8f5a23f85311 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:46.256687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95ff9-be23-44de-866d-8f5a23f85311 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:46.306078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:46.306734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.307010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.307280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.322579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227a6a-c2cb-4deb-b463-0e7615f6d3f7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.323220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227a6a-c2cb-4deb-b463-0e7615f6d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.323703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227a6a-c2cb-4deb-b463-0e7615f6d3f7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.324325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:46.325796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 243/483] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:46.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:46.331463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:46.332295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 241/484] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:46.334761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.335266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.335793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.336992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:46.376127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:46.377719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:46.396811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:46.466143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802b9c-f986-4a1c-ad74-ee0bd3002bba None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:46.480544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:46.517156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 244/485] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:46.527439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:46.528041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.528322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.528595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.529260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:46.529563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:46.530994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:46.539236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 242/486] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:46.547762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:46.556466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:46.734881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683f5ce-ea8b-4696-81c9-0b7130439bba None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:46.753750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4aa5f-ee7c-40fc-8a13-a30b66d55cd5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:46.771094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:46.785078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321db53-d8eb-4c7e-a5fa-933411b3c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:46.785763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321db53-d8eb-4c7e-a5fa-933411b3c374 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.815906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321db53-d8eb-4c7e-a5fa-933411b3c374 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.815906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321db53-d8eb-4c7e-a5fa-933411b3c374 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.816116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:46.816116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.816116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.816116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.816390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321db53-d8eb-4c7e-a5fa-933411b3c374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:46.816461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5321db53-d8eb-4c7e-a5fa-933411b3c374 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:46.816528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:46.817375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:46.818180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:46.819786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:46.819930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 243/487] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:46.819930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a4b01-71e5-47dc-8bff-abc5e29f52ab None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.819930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a4b01-71e5-47dc-8bff-abc5e29f52ab None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.819930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a4b01-71e5-47dc-8bff-abc5e29f52ab None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.819930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 244/488] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:46.819930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a4b01-71e5-47dc-8bff-abc5e29f52ab None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-240c8fc7-7b87-4733-a9e8-98323c292942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:46.822753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:46.824902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:46.827602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:46.827602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:46.827602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:46.827602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 245/489] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:46.833154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.833154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.833154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.848470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:46.849654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:46.879845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:46.926760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 246/490] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:46.928265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:46.929585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a4b01-71e5-47dc-8bff-abc5e29f52ab None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:46.969505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:46.970942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:46.971902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:46.973059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:46.973885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:46.974343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:46.976145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:46.985904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 245/491] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:46.987208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.032469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.051503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41556d-c59f-47fb-8870-6808196c652a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.088670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.089540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.089956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.090376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.091192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:47.091654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.095734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.097961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102336-1635-4fd3-9848-c123135adcfb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.112695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 247/492] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:47.113808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.136033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.141807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.141807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.141807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.146426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.149967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:47.155375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:47.156500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:47.157397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:47.161535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 246/493] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:47.162621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.228469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1e9c-54bb-4e80-a70c-cc837ddebeab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.269502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.276603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84534294-0bf4-494c-b2e1-9d0b3a4d05ef None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.287383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.288633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.289330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.289764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.305156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.306594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.312935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.313745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.313990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.314255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:47.321305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:47.322242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:47.323758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:47.323758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:47.323758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:47.323925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.324003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 248/494] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:47.324003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.325758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d4f11f94-5523-4fae-ac27-735c2ebffd50 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:47.341859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:47.342895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:47.343850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 247/495] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:47.344410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aaaeb-2db0-40ac-b57d-619b8bf5c098 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.344474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aaaeb-2db0-40ac-b57d-619b8bf5c098 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.345245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aaaeb-2db0-40ac-b57d-619b8bf5c098 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.346714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 248/496] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:47.351406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aaaeb-2db0-40ac-b57d-619b8bf5c098 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.413317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.437126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a909-1fc2-4cc5-9ab8-0aefe0b0be4f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.459004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aaaeb-2db0-40ac-b57d-619b8bf5c098 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.470054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.470647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.470945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.471210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.471858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:47.472153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.473513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.481963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 249/497] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:47.482771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.494124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.494744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.495027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.495300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.495936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:47.496236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.497611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa33bf-dd2e-43e7-9e8f-d1d8040e9557 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.510299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 249/498] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:47.511726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21406233-23c4-4f79-a057-6f64961ad877 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.512141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21406233-23c4-4f79-a057-6f64961ad877 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.512573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21406233-23c4-4f79-a057-6f64961ad877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.529713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-21406233-23c4-4f79-a057-6f64961ad877 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:47.530932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-21406233-23c4-4f79-a057-6f64961ad877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:47.543630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.587301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67866e-e1c1-42fa-b418-dcf886d92717 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.598331 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 250/499] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:47.599610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4d2f-73ea-42df-9b59-25b5375176c8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.599892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4d2f-73ea-42df-9b59-25b5375176c8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.600162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4d2f-73ea-42df-9b59-25b5375176c8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.601507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 251/500] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:47.602039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4d2f-73ea-42df-9b59-25b5375176c8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.634939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.635541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.635821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.636091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.636871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:47.637178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.638606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.665197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.676943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 250/501] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:47.680064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.703815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4d2f-73ea-42df-9b59-25b5375176c8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.737272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.738101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.738378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.738648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.749498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.751246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.759778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-16e8fef5-ede6-4ce2-a15c-b64b4a0fcc3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:47.765842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:47.772532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 252/502] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.773379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.777470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8471b3a-7d23-4737-9524-70ff3dcdf23b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.786057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:47.787269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:47.811867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.812415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.812685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.812956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.819884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:47.821337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:47.833786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:47.834938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 251/503] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:47.836046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.857710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 253/504] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:47.859467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.894055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.935870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bf9f6-1276-4554-9948-0b0762cac791 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.956373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a2e7-a3d8-40c7-a96c-8d3d56590aa8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:47.969922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.970543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.970846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.971120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:47.980759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:47.984636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:47.987662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:47.988576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:47.989373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 252/505] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:47.990048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:47.995120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:47.995722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:47.995994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:47.996284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.062692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.064068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.092140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e164-721f-46f5-83f8-eff41768f10a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.112410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.130175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.130942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.131262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.131529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.132407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:48.132527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.133814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.145289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 253/506] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:48.145289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.170188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 254/507] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:48.171593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.204361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.246040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a983d23-11a1-4521-8e42-e01c2b2d0b23 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.274254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e56651-661c-4275-b154-1a972e35c83e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.280976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.281574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.281852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.282131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.282872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:48.283172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.284600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.301647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 254/508] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:48.302276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.314279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.314865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.315140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.316082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.326212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.327747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.344397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:48.344745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:48.345862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:48.346974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:48.346974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:48.346974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:48.346974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 255/509] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:48.346974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.413777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff4da-69a5-41b7-bc2d-16933be5a622 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.430722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.449072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199f4da-b92a-434e-8490-165d7abcbb1c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.455301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.456332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.456790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.457295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.458352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:48.458724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.460125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.468809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 255/510] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:48.469613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.482593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.483731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.484003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.484267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.491500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.492905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:48.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:48.505579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 256/511] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:48.506893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.565592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.569373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c1d49-4c1c-4324-9395-de0586b720bb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.602095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.602742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.603019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.603284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.609490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df8f3-fce9-4200-b5ca-d1cc3af92ae7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.613000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.614439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:48.630351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:48.631543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:48.632826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:48.632826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:48.632826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:48.632826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 256/512] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:48.632826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.646891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.647487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.647766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.648037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.660592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.715425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.717272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.732588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394ce21-4340-454e-bd34-8725a05b4cd2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.765233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.765827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.766103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.766370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.767023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:48.767320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.768681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.780521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 257/513] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:48.781511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.792642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.820076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 257/514] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:48.820827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.865171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:48.879062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9409c-09f8-44e2-963b-98a642201ac4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.913355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.913969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.914246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.914518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.920749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2c9f1-2ff6-4a61-a688-c8ff5cff402f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:48.924533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.925969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:48.940161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:48.941276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:48.942125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:48.942125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:48.942125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:48.942125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 258/515] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:48.942349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:48.955457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:48.956055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:48.956332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:48.956601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:48.957320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:48.957623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:48.959083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:48.995583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 258/516] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:48.996319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.008475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.047429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73301bb-0b46-43b2-952f-f94b5a533b07 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.081055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.081574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.081849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.082116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.082789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:49.083091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.085126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.091810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.095447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507dce3-3c72-4e87-9096-03864498df7b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.097994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 259/517] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:49.098837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.133249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.133848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.134123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.134391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.135051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.136491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.195514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e15a1-ebae-4f0e-be52-ddf1c13d23ca None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.218415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.229463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.230076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.230354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.230622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.240207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.241645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:49.257258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:49.258261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:49.259157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:49.259157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:49.259157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:49.259157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 260/518] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:49.259384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.282701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 259/519] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:52:48 2022] POST /identity/v3/users => generated 247 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:52:49.282701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.372463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81134bad-97d6-46d4-aee9-3b95de4bd415 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.402388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3d0e5-2c7b-41ef-afa0-a2ab400370d6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.411985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.412587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.412869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.413151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.413818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:49.414115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.415532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.429684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 261/520] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:49.430372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.452761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.453381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.453632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.453896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.469443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.492712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.537423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.538913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.541408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1305-dde8-4f48-ac52-77efbcda6798 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.577956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.578579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.578948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.584458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.589240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.590726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:49.603292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:49.604389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:49.605254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 262/521] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:49.605750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.660698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 260/522] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:49.660698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.711017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.718363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27464d55-7325-47ff-8fc4-ba8d3d309219 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.719306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.752386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.753107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.753398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.753652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.754290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:49.754598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.756008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.765378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b14733-cbf4-49db-9965-9d1c13faf22f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.768060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 263/523] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:49.768910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.806716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.807343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.807623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.807901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.815850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.817505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:49.830540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:49.831702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:49.832599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 261/524] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:49.833168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.884355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24c476-a3d4-4b29-96bf-4732e2ffd4b0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.925091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.925091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.925091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.925091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.925530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:49.925530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.925530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:49.938403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e527-932a-43c6-b3dc-21811ae4b9c6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:49.948179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 264/525] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:49.948179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:49.957983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.967474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:49.978122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:49.978768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:49.979043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:49.979369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:49.990290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:49.991909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:50.007003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:50.008405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:50.009267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:50.009267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:50.009267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:50.009267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 262/526] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:50.009267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.057139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ac6a9-45b4-4243-aa14-7ea00901983d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.098119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.099420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.099703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.099978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.107960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.109419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.119837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c7c7f-a27a-4ef8-9086-e08f5d0a726e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:50.126429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:50.127359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:50.128219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 265/527] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:50.128697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.162399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.163036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.163316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.163586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.164314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:50.164612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.166071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.184324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 263/528] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:50.185058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.215109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:50.231509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:50.257963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd890-70ed-4a96-b558-ea53788abf12 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.289547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dc199-1f57-44b6-b4cc-c00707cb3921 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.293908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.294778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.295302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.295773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.296786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=07cef180b05347ebbcc6f3a6e97dac3a) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:50.297241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.298798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.314696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 266/529] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/role_assignments?user.id=07cef180b05347ebbcc6f3a6e97dac3a&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:50.314696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.326753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.327862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.328141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.328411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.329060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:50.329360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.330791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.343691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 264/530] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:50.344630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.458207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:50.482015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223db95-6a98-41df-a02c-3d5ef425e310 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.490945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af047b-8096-450a-8853-3319dbdc0b74 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.525076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.525752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.526054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.526303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.542768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.543711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.544060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.544425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.545881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:50.546553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.548037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.557749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:50.566244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 267/531] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:50.570957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.604339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.605912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.701965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b424112-84f4-4555-a08e-7446baccfb1b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.709091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:50.725564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 265/532] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:50.725824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.756852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.758007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.758438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.759001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.768813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.770402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:50.782784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:50.783882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:50.784984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:50.791070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 268/533] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:50.791070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.853165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:50.865173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bed8b-f6d4-4c8c-aa00-94e26dce2105 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.900783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f90902-4dd3-40a6-b88b-25dd1cf1c3d7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:50.908881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.909663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.910061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.910429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.911287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=f28dc00f9c744cc4b103b8e01d65f75f) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:50.911841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.913208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.919521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 266/534] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/role_assignments?user.id=f28dc00f9c744cc4b103b8e01d65f75f&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:50.921104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:50.946103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:50.947023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:50.947396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:50.947765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:50.960344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:50.962151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:50.979088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:50.987089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:50.987966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:50.987966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:50.987966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:50.987966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 269/535] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:50.994238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:51.001636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:51.057071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036ed72-70cf-4dba-9e04-588af8f1ddd8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:51.102693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:51.103329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.103591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.103864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.104887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:51.105191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:51.106939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:51.130624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 267/536] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:51.152777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:51.233406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c6e21-bfa1-4dc2-b406-20ddbf074fe5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:51.258494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:51.283110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378fbb0-91f3-4a2c-83d2-e7f6fc2b0197 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:51.295904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:51.324407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:51.325018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.325297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.325566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.326024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:51.326291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=54fff64dfcf8434d867c028a72c17a72) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:51.326630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:51.327315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.327794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.328086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:51.328458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.329357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:51.329849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:51.335491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59711128-aa94-4b9e-b37e-18e4cf0619bf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:51.342326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 268/537] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/role_assignments?user.id=54fff64dfcf8434d867c028a72c17a72&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:51.359650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 270/538] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:51.365509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:51.393391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072ddcb-4886-48ec-8a05-0328adffcef3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.393695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072ddcb-4886-48ec-8a05-0328adffcef3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.393944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072ddcb-4886-48ec-8a05-0328adffcef3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.395571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 271/539] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:52:51.400970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.401280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.401520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.425905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:51.429919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:51.566898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202c8c6-4e9d-4740-87e6-34fcf277afed None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:51.575409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 272/540] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:52:51.607119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:51.610253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:51.611052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.611331 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.611601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.621142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:51.622613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:51.635660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:51.636844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:51.637774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 269/541] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:51.649027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:51.726031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:51.730026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59e3df-95d5-4530-b917-2ef348108659 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:51.737080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:51.773459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:51.774099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.774381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.774650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.796560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:51.801355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:51.802241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:51.803375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:51.811197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 273/542] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:51.811521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00dd12-f7b5-43e8-9535-5db1cbcefd16 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:51.819076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:51.855201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:51.855813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:51.859902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:51.860193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:51.860909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:51.861215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:51.862684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:51.885229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 270/543] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:51.892646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.000537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:52.009158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2ea9-79b6-4f56-bef4-d1d671d76e44 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.046936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b137a6-9126-498c-a89d-9428f05cf7e8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.047483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.048088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.048511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.048805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.049527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=4337750a9fb940a387400d1f0ca9ba02) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:52.049836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.051303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.058007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 271/544] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/role_assignments?user.id=4337750a9fb940a387400d1f0ca9ba02&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:52.132752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.133385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.133661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.133931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.134628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:52.134957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.136461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.154456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 274/545] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:52.160491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.284509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f5711-0a23-46d2-b9d2-bc2578887406 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.320950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.321585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.321869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.322144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.332886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.334346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-91455524-5104-4ff9-aece-6806d8915e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:52.350210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:52.351318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:52.352173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:52.352173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:52.352173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:52.352173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 272/546] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:52.359168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.391218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:52.472420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12099726-da6d-497a-b837-42685c7286b7 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.472741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12099726-da6d-497a-b837-42685c7286b7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.472953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12099726-da6d-497a-b837-42685c7286b7 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.474526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 273/547] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:52.475479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f44d9e-a3a7-4bbd-9beb-123b6253428a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.480340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.480613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.480882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.515892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:52.518493 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:52.526882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:52.547416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.548041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.548333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.548554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.549208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:52.549523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.552029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.576571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 275/548] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:52.584264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.637224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 274/549] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:52.682820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.703523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:52.721684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36165-d27a-41a2-ab6f-1a34902d9743 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.759383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.760023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.760298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.760564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.768151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.769643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:52.781342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:52.782470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:52.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 276/550] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:52.796131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.822700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42be52e-974f-467c-a55d-c0d60b75af54 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.869592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.870223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.870499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.870791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.889581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:52.896171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:52.905465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:52.906499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:52.907669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 275/551] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:52.913304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.972723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0935b03-0bdb-4200-8767-fbead3e02358 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.027378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.028623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.029354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.030142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.031390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:53.032211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.034233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.059383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fac33-525f-4a06-94f2-7b8e4783b1d5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.068670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.073963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 277/552] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:53.082091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.113753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.114361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.114634 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.119985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.121159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:53.121465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.122886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.141052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 276/553] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:53.150562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.220675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ca640-0d01-4399-b300-634ad0163f14 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.255581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.256101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.256372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.256692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.325404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.327088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.328645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.338156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.437178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 278/554] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:52:53.440015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.455743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724ff4c-e557-4e56-95b8-d98508b5eb2c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.513822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.515307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.515583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.515852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.529456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.531998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c7136fb5-55ba-47f1-8c40-d80f66f28d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:53.552015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:53.553118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:53.554010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:53.554010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:53.554010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:53.554746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 277/555] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:53.555997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05a648-ab3f-49b9-ab36-edaf9194ea5f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.556278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05a648-ab3f-49b9-ab36-edaf9194ea5f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.556550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05a648-ab3f-49b9-ab36-edaf9194ea5f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.559222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 278/556] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:53.559222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05a648-ab3f-49b9-ab36-edaf9194ea5f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.585426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.590286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc00b40-706d-4fe2-bc37-ffec558614ec None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.640725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.641350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.641623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.641889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.656410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.658129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.676848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05a648-ab3f-49b9-ab36-edaf9194ea5f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-af1a7d4f-40c9-4e0a-bc4f-a198d26208f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:53.678732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:53.679681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:53.681032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:53.681032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:53.681032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:53.690861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 279/557] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:53.692360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.692641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.692914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.701152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.718043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:53.719840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:53.723519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.724413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.724941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.725436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.726275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:53.726836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.728575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.742186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 279/558] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:53.743336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.811309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 280/559] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:53.812630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.870103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0faf3bf-1598-47f9-a5a2-835858f19fcb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.906684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.907333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.907612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.907878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.908510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:53.908803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.910249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.913877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.923401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 280/560] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:53.924583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.939868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee63d15-47c9-4171-867d-e263573a1f8e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.963958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:53.975621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.976257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.976518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.976796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.984552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.986054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-06075c54-0320-405d-ae8d-e3f0640e74c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:53.997398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:53.999079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:52:54.000285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:54.003929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 281/561] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:54.005110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce123b-0b9e-48fe-a3b7-fd92f424c0b7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.005397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce123b-0b9e-48fe-a3b7-fd92f424c0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.005668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce123b-0b9e-48fe-a3b7-fd92f424c0b7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.007079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 282/562] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:54.007630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce123b-0b9e-48fe-a3b7-fd92f424c0b7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.038811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987962-929e-4ae0-8428-be4b9748e849 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.074955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.075610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.075844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.076038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.091205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:54.096426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:54.097371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:54.098217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 281/563] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:54.098968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.120444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce123b-0b9e-48fe-a3b7-fd92f424c0b7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.155958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.156576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.156858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.157126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.164261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.165726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d8cb11e1-2063-46bd-9611-1f107ef280b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:54.177448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:54.178394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:54.179334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 283/564] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:54.180174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.180174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.180174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.187569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.200758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:54.201981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:54.205920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.212743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155e10-5a7c-4458-b2f7-6fc892c6c97c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.252886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.253514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.254008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.254281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.255418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:54.255725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.257147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.282908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 284/565] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:54.282908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.289769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 282/566] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:54.290703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.384675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8229c-b3bc-40b4-b05f-c0d76d5ae0c1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.400935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be5cba-7142-4e2a-baf9-e535677b9f3c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.415704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.421578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.422190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.422456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.422757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.423424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:54.423725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.425126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.437307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 285/567] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:54.438812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.451424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.452029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.452579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.452579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.453243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:54.453543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.455002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.471738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 283/568] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:54.471738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.484809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.554353 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315162d-b08a-4d28-8a2b-cd7d1c952d9b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.577863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe66dc5-8424-4684-9f46-d9065bbc01a9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.596319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.596949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.597222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.597484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.608116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.609537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.613089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.613724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.614006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.614278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:54.622944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:54.624406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:54.625565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:54.625565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:54.625565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:54.625565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 286/569] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:54.625565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.627351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:54.634169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:54.640836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 284/570] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:54.641790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.683089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.711513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.735711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1fc5-6720-4b01-b346-c49baa4fca88 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.740568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f40d4b-6cba-4661-b80e-fb699dd5af81 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.770352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.771009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.771290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.771557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.773712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.774293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.774567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.774865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.775569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=69b58d64a6e04c1487f76e5ea26c74a9) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:54.775866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.777273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f40a4-528f-4a1c-9fbc-7b8d2ca04b03 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.781959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.782520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 285/571] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/role_assignments?user.id=69b58d64a6e04c1487f76e5ea26c74a9&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:54.783413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.783793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7308a3b-0cbd-4dcd-98db-58117de53f89 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.784054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7308a3b-0cbd-4dcd-98db-58117de53f89 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.784332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7308a3b-0cbd-4dcd-98db-58117de53f89 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.785536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 286/572] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:54.786247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7308a3b-0cbd-4dcd-98db-58117de53f89 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:54.813737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:54.815012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:54.816131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:54.816131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:54.816131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:54.821860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 287/573] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:54.821860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:54.914121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7308a3b-0cbd-4dcd-98db-58117de53f89 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.929835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.933791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcc7df-c9cd-493e-a4b2-c3a4d5a0d31f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:54.958689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.959297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.959571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.959845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.960478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:54.960772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.962129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020ec37-4f13-47aa-8d77-86e671399a07 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.964658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:54.971398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:54.972008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.972286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.972555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.973211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:54.973511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:54.974931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 287/574] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:54.975069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:54.980844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:54.981332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:54.981601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:54.990619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 288/575] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:54.991547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.079679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:55.080869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:55.142355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a75e5-6a29-4bc2-9fb4-4bf84b5a6a30 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.150486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 288/576] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:55.151406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.188665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.189217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.189565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.189993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.190605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:55.191002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.192457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.207033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 289/577] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:55.207883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.235605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:55.276413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae4323-6355-4de0-af2b-7a8aaa295a4e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.305138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:55.314217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829672a-9509-442d-9698-5990652049f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.314943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829672a-9509-442d-9698-5990652049f3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.315279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829672a-9509-442d-9698-5990652049f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.326254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829672a-9509-442d-9698-5990652049f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.326409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e17b1-3e76-4690-a0df-80f393ecbfe5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.326505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829672a-9509-442d-9698-5990652049f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.326505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829672a-9509-442d-9698-5990652049f3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7829672a-9509-442d-9698-5990652049f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:55.336205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:55.337277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 289/578] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:55.343763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829672a-9509-442d-9698-5990652049f3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.359309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.360176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.360954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.361375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.371968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.373727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:55.390720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:55.391762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:55.392642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:55.392642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:55.392642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:55.392642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 290/579] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:55.392642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.463772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829672a-9509-442d-9698-5990652049f3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.468462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:55.499449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5de42-dca2-4a90-a938-f06b15c0755b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.501498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.502272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.502647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.503034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.513633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:55.518869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:55.520033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:55.521104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:55.521827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 290/580] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:55.522784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.542416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.543104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.543386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.543660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.551808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.553219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:55.566081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:55.567225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:52:55.568393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:55.568917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 291/581] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:55.569320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.577141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:55.636070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb0268-ce74-4461-8610-5c3bcbb9563a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.680213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.680954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.681325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.681784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.682568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:55.682986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.684513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.685079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135840-e197-49fa-9eab-9b227b817e92 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.696372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 291/582] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:55.702539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.715450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:55.721437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.722044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.722324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.722592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.723255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:55.723554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.724910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.741309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 292/583] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:55.741309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.809571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fae1e-3559-433a-879e-672b0231c3bc None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.817525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:55.845932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.846542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.846846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.847117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.847786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:55.848068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.849498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.851171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10d539-dd02-4716-9362-3f37f98b21d6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:55.857376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 292/584] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:55.858188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.893857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:55.894582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:55.894956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:55.895291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:55.896090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:55.896460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:55.898003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:55.921271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 293/585] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:55.921271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:55.977708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f367f-bcc7-4003-a1e6-d005a4be77df None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.009899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.024845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.025613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.025960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.026304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/07cef180b05347ebbcc6f3a6e97dac3a/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.041365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=07cef180b05347ebbcc6f3a6e97dac3a, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.042832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.063682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980704d-7a37-441f-b005-4b130dabf3a1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.096914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.097514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.097791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.098060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.104889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.106310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:56.117434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:56.123111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 294/586] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:52:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:56.124670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.186688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 293/587] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:55 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/07cef180b05347ebbcc6f3a6e97dac3a/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:56.186688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.206102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.252808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a8e89-188b-4728-bee8-be67c344741b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.266691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.294041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.294723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.295034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.295365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.307745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.309585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:56.327687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:56.328720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:56.329611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:56.329611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:56.329611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:56.333776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 295/588] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:56.334779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.337014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776025f0-106c-4811-99a7-5ff9fe15ffd0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.382453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.383525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.384032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.384561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.400240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.402669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:56.429035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:56.429988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:56.431013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:56.431013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:56.431013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:56.431013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 294/589] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:56.431637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.478821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ce4aa-2e11-41fc-a60e-45597d1a1c34 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.515021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.519404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.520143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.520608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.520983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.521866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:56.522271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.523865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.548333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 296/590] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:56.549423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.559104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5686-e72e-45d5-8355-79cb1a62151f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.580906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.596599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.597373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.598373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.598373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.599280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:56.599941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.602494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.622531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 295/591] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:56.623741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.679017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fed25e-0c8e-4b39-9498-7c849a72e340 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.729261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.730987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.731418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.731859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.733480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:56.734180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.735885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.755920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 297/592] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:56.755920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.763534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3af82-f05e-4e4e-bb23-bf764d99bb8c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.801007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.808323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.808936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.809201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.809471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.819954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.821381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:56.836455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:56.837465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:56.838374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:56.838374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:56.838374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:56.842750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 296/593] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:56.843597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.846630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:56.865923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae87e7-7b0f-470b-b2ea-5a7fa1b109c4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:56.902982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:56.903715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.903987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.904254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.913008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:56.914583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-423ab571-5001-4c9b-80a4-713d51c0b084 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:56.926873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:56.933884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 298/594] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c821b81-4d8f-4f50-b4d5-9ad6ca038ed6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:56.935080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c821b81-4d8f-4f50-b4d5-9ad6ca038ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:56.936359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c821b81-4d8f-4f50-b4d5-9ad6ca038ed6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:56.936359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 299/595] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:56.936359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c821b81-4d8f-4f50-b4d5-9ad6ca038ed6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:56.956253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc88090-7d37-4e86-8f8b-f43f7e7da225 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.027257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.027866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.028146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.028415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.035792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.037233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:57.049067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:57.049976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:57.050899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:57.054906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 297/596] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:57.055718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.070464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c821b81-4d8f-4f50-b4d5-9ad6ca038ed6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.101970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.106285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.106919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.107199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.107497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.108202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:57.108500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.111911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814d819-d22e-4309-acaf-6c0554863392 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.119257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.123383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 300/597] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:57.124136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.124423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.124696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.143296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:57.144523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:57.167339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c75820-1cf1-4a86-819c-a785948359b2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.206074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.206798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.207145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.207419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.208136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:57.208433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.209839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.214161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 301/598] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:57.215176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.230472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 298/599] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:57.231542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.339264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.343309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1d82-4078-43f5-86eb-a11917f32ac9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.348959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c87c9f-dc68-4c05-84dc-0819a9702a84 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.379387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.383779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.384612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.385018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.385303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.385625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.385900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.386129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.386243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.386881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:57.387181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.388610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.395044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.396708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.400899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 302/600] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:57.401710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:57.409920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:57.410913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:57.411976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:57.418411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 299/601] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:57.419490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.519154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aed530-87b1-46fc-bda6-7131ff5cf5cf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.526281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268d6d-63d0-4da5-90f2-1c476efb4135 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.554795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.555733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.556282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.556554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.561381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.562002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.562279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.562543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.567939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.569484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.572832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:57.578269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:57.579245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 300/602] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:57.585217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:57.585987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:57.587387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:57.588338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:57.588338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:57.588338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:57.588338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 303/603] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:57.588338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.614596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.632251 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.696244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b04b4-2ba7-4e4e-9383-2e2bf65c5b01 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.703447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672ffe-35b6-4ffd-8761-e69447fb58c9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.733823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.734468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.734769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.735046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/f28dc00f9c744cc4b103b8e01d65f75f/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.740050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.740752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.741160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.741525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.742309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:57.742743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.744264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.750689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=f28dc00f9c744cc4b103b8e01d65f75f, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.752102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.758123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 304/604] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:57.759077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.821942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 301/605] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:57 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/f28dc00f9c744cc4b103b8e01d65f75f/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:57.823095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.886460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a688-619e-4497-86ec-6de57123d059 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:57.889176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.927672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:57.940445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:57.941646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:57.942130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:57.942548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:57.943594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:57.944090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:57.946423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:57.960383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 305/606] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:57.961792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:57.966200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ad54e-91e3-4f30-874e-aac6ec372dc0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.005568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.006166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.006446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.006749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.007390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:58.007743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.009496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.023422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 302/607] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:58.024285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.086542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56584d16-a65c-4883-a712-7e2caeafa9c3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.124530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.125619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.125996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.126371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/54fff64dfcf8434d867c028a72c17a72/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.143887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=54fff64dfcf8434d867c028a72c17a72, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.145396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.160280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dcac3-c10f-4be6-9e8c-86fb55b922ec None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.192316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:58.198122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.198787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.199120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.199391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.210128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.211616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:58.225730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:58.226883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:58.227823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:58.227823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:58.227823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:58.227823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 303/608] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:58.228047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.254689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 306/609] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:57 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/54fff64dfcf8434d867c028a72c17a72/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:58.257086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.303127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:58.371573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb108d0a-40fb-42c1-b3be-8367fb7d4331 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.405613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8694dee-3863-4534-9cb4-6bc4f014b804 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.412204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.412993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.413376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.413750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.423115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.423632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-acb17fa5-1250-406a-bd62-1b1417664da6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:52:58.441602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:58.442778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:52:58.443697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:58.449739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 304/610] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:58.449739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd9033-9d8b-4eb7-a8ce-6cfd779e6163 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.449739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd9033-9d8b-4eb7-a8ce-6cfd779e6163 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.449739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd9033-9d8b-4eb7-a8ce-6cfd779e6163 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.449739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 305/611] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:58.451587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd9033-9d8b-4eb7-a8ce-6cfd779e6163 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.455076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.456499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.456499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.456499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.457289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:58.457659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.459318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfe22c-30f3-4a5e-976d-8de09f92eaa1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.472952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 307/612] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:58.474280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.474628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.474992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.493298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:58.494480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:58.516218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:58.572204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 308/613] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:52:58.573153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.576614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:58.600256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd9033-9d8b-4eb7-a8ce-6cfd779e6163 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.640474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.641240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.641612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.641989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.642808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:58.643227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.644743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.657760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 306/614] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:58.658759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.682260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711460a6-d0ad-4bff-8f04-506209226971 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.719412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.719787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.720126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.720401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.727976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.729422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:58.741985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:58.743186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:52:58.744495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:58.750238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 309/615] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:58.751160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.766996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa98d52-19bd-45c6-9a53-014f5762ec4b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.774980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:58.802693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.803649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.804069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.804507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.816092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:52:58.818885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:52:58.822168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:58.823400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:52:58.824461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:58.830932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 307/616] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:58.831941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:58.871550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccb7c9-4927-437d-a554-5534bab1f487 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.928322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.928322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.928322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.928322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/4337750a9fb940a387400d1f0ca9ba02/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.940058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=4337750a9fb940a387400d1f0ca9ba02, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.942297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:58.947622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4866-fea5-4c44-8e51-0270e3da91fa None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:58.983931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:58.984614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:58.984969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:58.985308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:58.986116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:58.986499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:58.988033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.008913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 308/617] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 10 22:52:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:59.014450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.036587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:59.053928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 310/618] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:52:58 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/4337750a9fb940a387400d1f0ca9ba02/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:52:59.055167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.184029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf579-8837-41f0-ba80-73f9c637fa4e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:59.235960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:59.236607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:59.236844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:59.237118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:59.237789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:59.238091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:59.239501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.257713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 309/619] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:59.265516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.270932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8f87-0c70-4859-927c-d952d24ce0f2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:59.288001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:59.308362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:59.308994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:59.309272 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:59.309541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:59.320145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:59.321578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:59.338365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:59.341316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:59.342795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:59.342795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:59.342795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:52:59.342795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 311/620] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:59.345487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.374145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:59.395633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcc32-9dd1-459c-8527-2461a017a2a9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:59.436049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:59.437139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:59.437650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:59.438125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:59.452104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:59.453954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:52:59.470398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:59.471471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:52:59.472375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:52:59.472375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:52:59.472375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:59.479185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 310/621] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:52:59.481014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.600625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dedfef-dc5f-4101-be52-13520d4b1b21 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:59.646535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:59.648078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:59.648590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:59.649212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:59.650074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:59.650624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:59.652239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.654085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:59.672114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 311/622] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:52:59.672760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf709eb8-e69d-47cb-ae7d-e3016eb1115b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:59.674193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.798320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ff114-748c-4109-88a4-4988ed5ef91b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:59.799361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:59.799985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:59.800268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:59.800536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:59.801180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:52:59.801479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:59.803285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.833065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 312/623] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:52:59.841042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:59.853071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:59.853710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:59.853986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:59.854259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:59.867299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:59.868769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:52:59.885066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:52:59.886114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:52:59.887406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 312/624] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:52:59.889421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:59.894309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.003877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.007669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cdff2-bd08-4c1e-b16a-7fa95c46934a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.047417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750067-2c77-4a19-96ec-6849ed68e0e1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.050787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.051600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.052028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.052608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.065008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.067350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-13c3a38a-829a-45fa-a499-ceebd1884c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:00.083355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:00.084491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:00.085627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:00.086614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.087382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.087792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.088170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.088355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 313/625] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:00.089024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:00.089434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.090057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d61bd-46a9-44a9-9fb3-993692e77375 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.090497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d61bd-46a9-44a9-9fb3-993692e77375 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.090950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d61bd-46a9-44a9-9fb3-993692e77375 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.091325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f22e6-7866-4b52-8e24-25280f700321 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.093487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 314/626] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:00.094528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d61bd-46a9-44a9-9fb3-993692e77375 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.121155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 313/627] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:52:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:00.121155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.121155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.121155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.139057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:00.140537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:00.209999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.213104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 314/628] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:00.214361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.222099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d61bd-46a9-44a9-9fb3-993692e77375 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.261586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.269365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.272958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.272958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.272958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.272958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:00.272958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.273778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.292015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 315/629] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:00.293204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.324117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819613d-2e50-469b-b609-bd0452761c75 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.365962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.366578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.366885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.367156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.407871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7e3c-5a1e-4c36-a1a8-6a24500978a4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.448043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.450476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.461191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.462253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.462606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.462982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.475179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:00.480736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:00.481917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:00.483044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:00.484846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 316/630] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:00.487283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.500313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.512846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.575267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 315/631] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:00.576486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.603857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c05d9-afdf-4c6d-a3fb-6c76771ca2dc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.639505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994b29f-48eb-4666-aeed-199b319adbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.640355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994b29f-48eb-4666-aeed-199b319adbbf None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.640727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994b29f-48eb-4666-aeed-199b319adbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.641124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994b29f-48eb-4666-aeed-199b319adbbf None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/69b58d64a6e04c1487f76e5ea26c74a9/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.656872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994b29f-48eb-4666-aeed-199b319adbbf None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=69b58d64a6e04c1487f76e5ea26c74a9, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.658456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994b29f-48eb-4666-aeed-199b319adbbf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.686559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506cbda-ff6f-4291-98eb-3effec254761 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.721177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.721778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.722051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.722319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.723018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:00.723335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.724695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.733739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 316/632] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:00.735209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.739889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.752950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 317/633] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:00 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/69b58d64a6e04c1487f76e5ea26c74a9/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:00.754726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c6f8a-f75c-4817-af5f-e9025025f3e5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.755012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c6f8a-f75c-4817-af5f-e9025025f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.755283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c6f8a-f75c-4817-af5f-e9025025f3e5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.760802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 318/634] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:00.760802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c6f8a-f75c-4817-af5f-e9025025f3e5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:00.834147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:00.879118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c6f8a-f75c-4817-af5f-e9025025f3e5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.883373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671690e-046a-4940-908d-80285af0cb77 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:00.916827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.917597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.917936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.918266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.929999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.931665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.935539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:00.936403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.936821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.937256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5b07eba0-7001-4830-9b8f-017eb34b8bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:00.949370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:00.950586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:00.952348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:00.952348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:00.952348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:00.953767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 319/635] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:00.955510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.956014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.956436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.957738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:00.959451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:00.975549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:00.976793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:00.977949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:00.977949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:00.977949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:00.978142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:00.978142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:00.982805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 317/636] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:00.983723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.052504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.073779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 320/637] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:01.077898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.100130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5f8a0-7db2-48b1-b5e0-dc7eb2dae531 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.131966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.139470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.140159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.142490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.142490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.142490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:01.142490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.143462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.156830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 318/638] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:01.157792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.219640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c050902-d023-49e1-a539-3b8c4d4fb25c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.268898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.269519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.269813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.270058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.270681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a28614-e5ce-4436-bec9-1b13f1dfd70d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.271118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:01.271118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.272647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.290798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 321/639] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:01.291720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.292078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.315945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.317154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.317433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.317702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.333023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:01.338752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:01.341201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 319/640] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:01.343902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.399218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.404021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e256-b099-4ace-a7fe-45bab2e8827c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.447286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.447741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.448014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.448243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.456036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.457455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.466112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004a755-5f46-473e-9d93-9c68e04e0fc6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:01.469491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:01.470499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 322/641] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:01.471532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.509102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.509317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.509573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.509853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.517547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.519036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9fbfa0a7-9416-47d9-bc5b-13e7f8d3c65b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:01.530221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:01.531401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:01.532380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 320/642] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:01.532950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79153509-7e53-453b-bb67-62dc36fc61fa None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.533406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79153509-7e53-453b-bb67-62dc36fc61fa None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.533406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79153509-7e53-453b-bb67-62dc36fc61fa None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.534807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 321/643] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:01.539211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79153509-7e53-453b-bb67-62dc36fc61fa None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.581955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.591220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054135-6e0f-4282-8c2c-07f222a50330 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.626768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.627380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.627655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.627933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.628592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:01.628922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.630293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.640358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 323/644] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:01.641161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.660746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79153509-7e53-453b-bb67-62dc36fc61fa None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.667170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.697191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.697784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.698058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.698386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.699074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:01.699373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.700802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f0843-b061-4ce4-b473-390a63aafe7f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.723459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 322/645] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:01.723459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.723459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.723459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.739811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:01.741150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:01.774521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61393d5e-c123-4902-bf6e-939f20540b3a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.810269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 323/646] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:01.811969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.821198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.822072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.822551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.823055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.823951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:01.824549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.826242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.840617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 324/647] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:01.841758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.842896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.919290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b210fa-3433-48c0-aeb9-b60d97446d1e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.932911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:01.953904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163d6c-b822-466d-8641-6c47b46a4139 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.961425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.962607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.962922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.963188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.974828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.976269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.990603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.991260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:01.992081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:02.000305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:02.001469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:02.001469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:02.001469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:02.001647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.001647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.001786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 324/648] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:02.001786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.072671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.074112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.080702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.106683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f356e32-da3e-4f9a-be74-d957c4b885c6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.144401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.145049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.145323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.145589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.155773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:02.157034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:02.161275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:02.162455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:02.163374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 325/649] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:02.164065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.190189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 325/650] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:02.190189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.269988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386f081-698e-4984-95b1-5f40a208d276 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.296137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e786a8-8688-4e13-b4ba-01e444db5e65 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.306704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.307319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.307549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.307823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.308546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=07cef180b05347ebbcc6f3a6e97dac3a) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:02.308848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.310307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.313403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.316772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 326/651] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/role_assignments?user.id=07cef180b05347ebbcc6f3a6e97dac3a&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:02.317609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.349424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.350623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.351034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.351423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.352197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:02.352600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.354231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.375451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 326/652] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:02.376728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.438572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.459058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7ec35-4f2b-45ed-a20b-facadca6c1fb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.507047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.507760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.508107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.508450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.509238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:02.509597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.511107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.516116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8002a50-ff79-4bb6-ab08-4844e9121d0d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.522828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 327/653] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:02.523677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.565510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.567703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.567703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.567703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.582127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.583780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.599519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:02.603415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:02.604372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:02.605361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:02.605361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:02.605361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:02.605361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 327/654] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:02.605597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.650350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc78d4-d0bd-468d-bf03-1fc92dfdcbde None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.695156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.696672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.696957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.697231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.707504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.709725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.720140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:02.726755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:02.727834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:53:02.728818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:02.729386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317d1ca-e3d8-498c-b426-674d55eb7db4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.733001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 328/655] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:02.734524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.768998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.769653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.769928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.770194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.784977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.786547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:02.801431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:02.802513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:02.804154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:02.804154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:02.804154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:02.804154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 328/656] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:02.804154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.877156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3485d-e89b-47c5-91d0-3ef88068c5ba None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.908124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:02.917689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab848dd-cf0f-408b-aeb7-e79f8ab5f98b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:02.919583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.920723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.920723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.920967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.921453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:02.921749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.923160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.938832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 329/657] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:02.938832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.958343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:02.958984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.959263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.960844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.960844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:02.960844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:02.962021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d6ee4-b1fa-406c-ab04-ee063cc30b48 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:02.985632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 329/658] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:02.985632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d851c38-f738-46c2-a94a-656a012de765 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:02.985632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d851c38-f738-46c2-a94a-656a012de765 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:02.985632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d851c38-f738-46c2-a94a-656a012de765 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:02.985632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 330/659] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:02.985632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d851c38-f738-46c2-a94a-656a012de765 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:02.988933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.049385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67b3a9-3055-4db9-a36f-1bb069b4fbb9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.083438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.084035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.084315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.084588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.085221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:03.085563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.086920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.092914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d851c38-f738-46c2-a94a-656a012de765 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.103365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 330/660] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:03.103365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.132341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.133056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.133403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.133722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.141252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.142770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.147911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d1c2beff-1c0b-44c2-8d59-646e74837b13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:03.155791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:03.161634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 331/661] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.162581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.163239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.179242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:03.180438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:03.212446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc686c5-7b74-47a0-ad61-6d54833c50ca None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.238572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.249483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.264474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.264607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 332/662] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:03.264703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.264703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.264876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.337165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.338615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.355136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f58e2-386c-482e-a5c3-e89e5bc57d62 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.387643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.390838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.391463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.391740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.392014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.399504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.400937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:03.412792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:03.413890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 333/663] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:03.415578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.456991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 331/664] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:03.457836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.473242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.520309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d861b75-9ff6-4106-9902-e208a4a5c060 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.556704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.557298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.557577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.557847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.558534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:03.558865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.560824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.565987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29037c7-a7ce-40b7-86ed-1a49dc56954c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.579205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 334/665] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:03.579205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.603756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.604379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.604759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.604934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.616129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:03.620949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:03.621899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:03.623305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 332/666] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:03.624011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.630671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.692560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67af75-5726-4dfb-a655-9aeb17355ea2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.705765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.727217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.727755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.728032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.728298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.729017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:03.729326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.730779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.731506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2cfc-fe1e-48e1-91aa-69ed19aa601a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.755822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 335/667] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:03.757075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.780693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.781311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.781588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.781860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.792278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.793712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:03.808062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:03.809090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:03.810063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:03.810063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:03.810063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:03.810063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 333/668] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:03.811310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.869932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.876248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301b271-1692-4821-984b-e6d70e07368f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.915474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.916072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.916346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.916633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.916948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d12a3-2344-4443-b54a-e550fa0abdfd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:03.923061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=f28dc00f9c744cc4b103b8e01d65f75f) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:03.923061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.923061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:03.934330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 336/669] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/role_assignments?user.id=f28dc00f9c744cc4b103b8e01d65f75f&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:03.934330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:03.978687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:03.979579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:03.981038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:03.981348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:03.981622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:03.982369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:03.982700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:03.984249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.000693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 334/670] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:04.001907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.065836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218d83-a105-4001-a102-01b9eebae2f4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.105125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.105705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.105982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.106247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.148281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12cab-31cb-47be-b2d2-f3dd6b02ea48 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.169832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.191022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.192657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.194307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.194974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.195255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.195529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.196395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:04.196504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.197822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.212037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 335/671] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:04.212899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.245728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.333837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45735c-a587-4ae6-863a-1b2bef1eb393 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.334965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 337/672] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:04.335816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.375775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.376407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.376817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.377163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.388317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.389921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:04.405771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:04.406903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:04.407808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:04.407808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:04.407808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:04.407808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 336/673] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:04.407808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.448850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.485976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f6f9b-5961-49f6-8f1d-1dfae71781c7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.526615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce839a2-c966-4e9f-a887-fd4fdae66924 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.533090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.533908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.534333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.534779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.537592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.545303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.547418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:04.559638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:04.561122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 338/674] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:04.562024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.569298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.569932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.570213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.570484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.581631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.583148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-aaaa52b8-ef3f-4321-873e-34b504f1aa01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:04.597636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:04.598612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:04.599585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:04.599585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:04.599585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:04.599585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 337/675] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:04.600265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f35c9-fefc-40c8-9e2f-24152ae33f8f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.600265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f35c9-fefc-40c8-9e2f-24152ae33f8f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.600576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f35c9-fefc-40c8-9e2f-24152ae33f8f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.605708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 338/676] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:04.605708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f35c9-fefc-40c8-9e2f-24152ae33f8f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.671693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dd722-4183-4235-91af-5bac9fceffb3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.688416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.707953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.708671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.708945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.709211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.709398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f35c9-fefc-40c8-9e2f-24152ae33f8f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.709846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:04.710140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.711689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e4b77-fb62-4125-8757-a5e002c4c412 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.728946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 339/677] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:04.730554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.731032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.731480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.745035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.745640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.745917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.746185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.746884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:04.747183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.748620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.750718 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:04.751925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:04.768927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 339/678] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:04.768927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.815386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.834179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 340/679] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:04.835472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.878832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753a2ee-52b3-4c97-a12b-3a77a8000775 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.914466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.915090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.915416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.915687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:04.916434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:04.916781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:04.918273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:04.929416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:04.938040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 340/680] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:04.939066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:04.952685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5c4d7-2438-4d54-8710-5bd3c63b8b53 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:04.994501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:04.995434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:04.995831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:04.996109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.005653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.007179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:05.019097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:05.020116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 341/681] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:05.021070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.048149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100730b-b5f5-4d19-a4f7-8cf66f24e020 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.075532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.089485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.090099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.090376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.090643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.091484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=54fff64dfcf8434d867c028a72c17a72) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:05.091687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.093094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.098910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 341/682] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:04 2022] GET /identity/v3/role_assignments?user.id=54fff64dfcf8434d867c028a72c17a72&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:05.099679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.155915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8fc8-8cc1-4f50-9eb7-6d7a34ff3dd5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.187986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.203923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.204578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.204859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.205131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.217081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:05.222463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:05.223688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:05.224712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 342/683] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:05.225425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.238184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6997201-5bae-4c04-9b99-fb80fb394256 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.278397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.280032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.280454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.280907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.288582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.290189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:05.302980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:05.303931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:05.304822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:05.308960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 342/684] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:05.309818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.347675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7b81-3d4f-4145-83e8-b8bdd3121b51 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.353194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.389247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.389852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.390125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.390393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.391150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:05.391448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.396466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.418400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 343/685] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:05.425665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.443842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002aab-41f2-4a5e-bc8b-9f22409e75c3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.473355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.485788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.486384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.486684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.486969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.487715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:05.488239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.490703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.501226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 343/686] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:05.502126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.539582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113d9ce-7bcb-42a1-8f8a-d2d159df5c1e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.575447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.576975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.577454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.577924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.578935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:05.579488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.581117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.594374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 344/687] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:05.595713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.611324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9af21-3448-46c2-8234-c78cf8e85cbf None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.627119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.647679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.648285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.649043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.649043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.705628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a406f4-58e7-4b4b-b1b9-ff6ba03dc6ef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.707022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.720833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.722332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.749162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.750219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.750494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.750969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.769479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.772106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:05.793956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:05.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:05.796363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:05.796363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:05.796363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:05.796363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 345/688] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:05.796765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.869874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 344/689] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:05.870921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.883425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:05.909347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d8a2-801f-4077-95cc-eef1a908f14e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:05.944565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:05.945270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:05.945768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:05.946104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:05.946914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=4337750a9fb940a387400d1f0ca9ba02) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:05.947285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:05.948764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:05.955829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 346/690] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/role_assignments?user.id=4337750a9fb940a387400d1f0ca9ba02&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:05.956814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:05.980452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.006159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e0c4-2684-4e73-a50c-170e8d183865 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.067388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329ff8-e397-4660-bbc8-518a0e0b34e9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.070439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.071091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.071370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.071643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.072294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:06.072594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.073979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.096372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 345/691] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:06.103606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.108879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.109527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.109800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.110064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.117710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.121554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.123063 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:06.140570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:06.143013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:06.145253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:06.145253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:06.145253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:06.145253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 347/692] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:06.152881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.244747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf518-731e-4557-a749-d5c579289519 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.292621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.293263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.293541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.293815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.296287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a23ff-6928-451c-9822-ec18dd030a26 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.304645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.306736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c4cfa48a-33dd-45d4-96c4-c4b3bfe1be67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:06.319695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:06.320715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 346/693] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98e62-7721-4eb3-b567-334d07753501 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.322593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98e62-7721-4eb3-b567-334d07753501 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.323330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98e62-7721-4eb3-b567-334d07753501 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.324312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 347/694] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:06.328929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa98e62-7721-4eb3-b567-334d07753501 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.339991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.347901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.348572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.348896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.349182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.354758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:06.354758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.354758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2d243-b630-4b4f-a72a-03a0ac13dc31 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.365360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 348/695] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:06.366603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.367541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.367821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.397694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:06.399573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:06.457233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.473804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa98e62-7721-4eb3-b567-334d07753501 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.486769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 349/696] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:06.487982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.514134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.514785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.515061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.515326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.516338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:06.516641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.518064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.539756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 348/697] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:06.545266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.612087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12730-600f-4041-ad6d-69f45b67406d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.615106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.649399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.649957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.650238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.650510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.654419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef450c-62af-4d41-9c31-9b71dd7df84e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.657968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.659526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:06.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:06.671744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 350/698] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:06.673252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.696034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.696746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.697018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.697284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.708184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.709618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.721396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:06.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:06.728170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:06.729151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:06.729151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:06.729151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:06.733419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 349/699] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:06.734311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.787030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506bc4c-c966-41c8-87b3-da3fc53a6f8e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.823192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.824146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.824650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.825120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.840431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7bfea-2c3f-489a-b24c-679f659dcf56 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:06.858841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:06.876772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:06.877381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:06.877695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:06.877967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:06.878647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:06.878974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.880392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.893620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 350/700] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:06.894360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:06.902784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:06.904493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:06.948554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.002608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a8c0f-a045-4c2f-8364-f15b6c1266df None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.014433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 351/701] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:07.015870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.042830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.043428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.043699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.043965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.045107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:07.045399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.046766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.063148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 351/702] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:07.064028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.099543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.128119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213874-9e4a-4b53-8922-acc7636b833f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.163626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.164573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.165162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e5785-d5d2-477e-b4c0-8788505b8dde None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.165843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.166335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.168020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=69b58d64a6e04c1487f76e5ea26c74a9) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:07.168581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.170822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.177813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 352/703] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/role_assignments?user.id=69b58d64a6e04c1487f76e5ea26c74a9&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:07.179047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.183028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.227433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.228078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.228355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.228622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.251110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.252565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:07.282033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:07.283349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:07.284482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:07.284482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:07.284482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:07.284482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 352/704] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:07.291976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.296198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c9c93-a856-4e23-b684-bb5a24bd8f19 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.334412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.335327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.336039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.336326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.336978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.338453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.415113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.434469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b0c45-6a83-4bff-93f7-364e9f33e562 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.471807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.472408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.472685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.472951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.473590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:07.473890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.475283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091824d0-b0ed-4149-958a-862f14f4f728 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.487804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 353/705] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:07.488936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3d8a-dca4-44bf-b833-d4023dc2b00e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.489215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3d8a-dca4-44bf-b833-d4023dc2b00e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.489478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3d8a-dca4-44bf-b833-d4023dc2b00e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.490811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 354/706] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:07.496878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.497150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.497481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.500787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 353/707] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:53:07 2022] POST /identity/v3/users => generated 254 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:07.502933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.538965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:07.540640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:07.581390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.633167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be6ea3-5c6e-444b-82c9-0ea02ab4dd26 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.638995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 355/708] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:07.665375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.693845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.694495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.694808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.695087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.705791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.707982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:07.723088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:07.724082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:07.724990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:07.729538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 354/709] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:07.730692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.780627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.816079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab80bd-eba0-4a49-8478-c44c89854baa None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.866595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.867630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.868263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.869179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.870266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.872626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.875501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845acb59-9bb1-4864-93e0-358c15ed27a3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:07.892636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:07.893710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:07.894586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:07.898611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 356/710] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:07.899937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:07.911053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:07.929233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:07.930246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:07.930599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:07.931199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:07.932422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:07.932806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:07.934898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:07.951797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 355/711] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:07.959318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:08.024891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8967e-dde2-426c-b900-38229e8f9442 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:08.060982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:08.061754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:08.062136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:08.062529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:08.063289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:08.063684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:08.065144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:08.070242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 357/712] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:08.073992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b33a02-3f4f-4ded-9f4a-55f3f2d8ca95 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:08.076348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:08.085412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:08.111871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:08.112770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:08.113189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:08.113639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:08.114502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=07cef180b05347ebbcc6f3a6e97dac3a) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:08.115093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:08.116691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1ead2-615f-4ce4-832d-0e5656f442e0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:08.130822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 356/713] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:07 2022] GET /identity/v3/role_assignments?user.id=07cef180b05347ebbcc6f3a6e97dac3a&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:08.203397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b613bcd-13ef-4184-b4a9-ff1720dfc0ea None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:08.242619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:08.243399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:08.243795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:08.244136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:08.244853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:08.245220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:08.246858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f39c33-8f1a-408b-8b91-14440a28fc88 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:08.254629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 358/714] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 10 22:53:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:08.521874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de5282-984f-4d59-bc01-2b02760a2e9f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:08.522184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de5282-984f-4d59-bc01-2b02760a2e9f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:08.522379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de5282-984f-4d59-bc01-2b02760a2e9f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:08.523967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 357/715] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:08.529250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:08.529732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:08.529825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:08.549750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:08.550996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:08.618606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:08.626999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 359/716] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:08.645295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de5282-984f-4d59-bc01-2b02760a2e9f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:08.710914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:08.790438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de5282-984f-4d59-bc01-2b02760a2e9f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:08.836901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:08.837593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:08.837876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:08.838145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] PATH_INFO: `/v3/services/network` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:08.838862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:08.840241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:08.857800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:08.858892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:08.860183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 358/717] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:08 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:08.865177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:08.974217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c883e6-cef8-41b9-9c9d-db4f46980642 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:09.010017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:09.010624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.010941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.011217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.011852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] RBAC: Adding query filter params (name=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:09.012387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:09.013845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631053bb-6789-428d-8a9c-533d9e2ad66e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:09.018856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 360/718] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:09.025594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:09.276987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a6cb1-9614-4dc1-9ed9-7f8e1d63911f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:09.288844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:09.320417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:09.321053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.321331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.321600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.322229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] RBAC: Adding query filter params (type=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:09.322525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:09.323919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9891a5c-5a6f-4344-9983-5c2b895ce764 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:09.329617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 359/719] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:09.902574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c2afc-fcdd-487c-ac48-deb1ad65fdb1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.903755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c2afc-fcdd-487c-ac48-deb1ad65fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.904253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c2afc-fcdd-487c-ac48-deb1ad65fdb1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.906050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 361/720] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:09.911547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab17fb7-5ae1-45f8-8497-593a9349c9e6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.912143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab17fb7-5ae1-45f8-8497-593a9349c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.912622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab17fb7-5ae1-45f8-8497-593a9349c9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.931412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8ab17fb7-5ae1-45f8-8497-593a9349c9e6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:09.932967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8ab17fb7-5ae1-45f8-8497-593a9349c9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:10.022408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 360/721] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:10.055600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c2afc-fcdd-487c-ac48-deb1ad65fdb1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.136484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:10.189117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c2afc-fcdd-487c-ac48-deb1ad65fdb1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.207446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741af055-12d6-466a-ac22-7bbd0f8526f9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.208086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741af055-12d6-466a-ac22-7bbd0f8526f9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.208581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741af055-12d6-466a-ac22-7bbd0f8526f9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.210405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 361/722] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:10.215696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.216363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.216845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.260022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:10.261649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:10.376006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.376639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.376912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.377179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.399418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:10.405135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:10.406210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:10.407176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:10.407867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 362/723] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:10.413073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.415089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 362/724] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:10.435935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.493322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:10.615303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537173a1-6c2f-483d-b940-cb4cd2afc35e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.627415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22935146-3ee7-451b-8a65-e29337520f59 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.685734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:10.690701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.691315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.698431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.698431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.698431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:10.698431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.698431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.709329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.709952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.710224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.710487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.711115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 363/725] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:10.713555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.726036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:10.739352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:10.740400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:10.741292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 363/726] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:10.746750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.870947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:10.879551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f8b0b-045d-405e-920a-41225afb96df None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.904452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193fad-eed3-45ac-bc86-c54f8091f027 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.923294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.923921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.924196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.924464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.934826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.936325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.941981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.942579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.942885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.943158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.943816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:10.944117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.945486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41988c-25ca-40e0-a097-d6fa94a6ffea None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e09ed831-68f5-4a68-9d42-a4a62330a15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:10.952828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:10.953869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:10.954829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:10.954829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:10.954829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:10.955013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 364/727] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:10.955677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049f306-0193-4772-93a9-f4655527d2a0 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.955963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049f306-0193-4772-93a9-f4655527d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.956234 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049f306-0193-4772-93a9-f4655527d2a0 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.957629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 364/728] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:10.959423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222700da-61af-4737-8488-4cfd44928c71 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.963739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 365/729] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:10.963968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222700da-61af-4737-8488-4cfd44928c71 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.963968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222700da-61af-4737-8488-4cfd44928c71 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.963968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 365/730] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:10.964193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049f306-0193-4772-93a9-f4655527d2a0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.964271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222700da-61af-4737-8488-4cfd44928c71 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.032148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.089608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049f306-0193-4772-93a9-f4655527d2a0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.089835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222700da-61af-4737-8488-4cfd44928c71 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.124180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.125058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.125338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.125594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.127036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.127634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.127905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.128261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.128983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:11.129435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.130880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d7ce2-ecf1-4ee1-baa7-7377e7d34513 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.137340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.139567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.143815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 366/731] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:11.145147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.145422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.145688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-52ad6bed-9048-445e-8672-4b8e23c88d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:11.157208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:11.158176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:11.159716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:11.159716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:11.159716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:11.159716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 366/732] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:11.160133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.160410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.160677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.165584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.172400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:11.173592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:11.179395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:11.180631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:11.244905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 367/733] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:11.245747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.274165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 367/734] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:11.274165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.306509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.377875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab469c-aace-43c1-90ea-96b9d0b260c6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.381795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19b4bc-9001-4699-a638-9e0e383fe86d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.418814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.419549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.419819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.420085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.420403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.420849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:11.421038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.421133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.421297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.421565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.422348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.435412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 368/735] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:11.435789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.436233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.437695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.440410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:11.452708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:11.458872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 368/736] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:11.459851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.535447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.546981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4598954-b7b1-4856-982a-3945025d63f2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.565267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dce959-b718-406f-8c5c-10acca042e2b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.583947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.584566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.584840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.585105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.596125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:11.600064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.600664 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.600942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.601214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:11.601322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:11.602182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:11.603065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:11.603694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.603764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 369/737] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:11.603764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.608457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.702163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42119ee1-722e-4246-abdf-232555fc502c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.735323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.735953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.736218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.736539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.744252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 369/738] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:11 2022] POST /identity/v3/services => generated 251 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:11.744562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.745531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.746114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:11.769319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:11.770392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:11.771390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 370/739] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:11.772095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.786407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.828212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:11.902090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc7c05-5902-4e84-9192-2d29ddf49bee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.908890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ee40c-3351-4405-8d5e-28915b824037 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:11.941763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.942537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.943155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.943427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.944294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:11.944597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.946017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfde6ab-7b9d-4314-bc4f-e052484faf66 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.948003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:11.948859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.949299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.949620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.950281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:11.950609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:11.952008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:11.963746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 371/740] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:11.964757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:11.965655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 370/741] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:11.967686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6e9a-7c08-4057-a521-325dda003088 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:11.968101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6e9a-7c08-4057-a521-325dda003088 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:11.968376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6e9a-7c08-4057-a521-325dda003088 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:11.969579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 371/742] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:11.974884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6e9a-7c08-4057-a521-325dda003088 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.041826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.075394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbf8f2-08ee-4971-8c89-3f504854861a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.081631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6e9a-7c08-4057-a521-325dda003088 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.092629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.110624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.111269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.111560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.112059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.112924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:12.113657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.115188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0d863-fc5b-4fc6-9304-10ceea410073 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.117803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.118431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.118743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.119005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.129978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.131549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.138307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 372/743] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:12.138307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.138307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.138307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:12.152128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:12.153155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:12.158633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:12.158633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:12.158633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:12.158851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:12.158926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 372/744] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:12.158926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.159059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:12.226605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 373/745] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:53:12.227474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.277274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b6a3a-89d5-4790-8676-3dbb4f93eddb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.295375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.314294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.314934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.315668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.315668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.344651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960eb55-4db4-4adf-98fc-0be8efc3452f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.353123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.381313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.381901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.382272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.382544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.394025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.396003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.458318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.459954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.504892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 373/746] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:12.505752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.524450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.567975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 374/747] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:53:12.569105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.576276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.618087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663f526-df4e-4a2b-afa7-b4ac4146dfa2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.656726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.657477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.657855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.658201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.658928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:12.659426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.660945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.674050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 374/748] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:12.674050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.678836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2f0be-64dc-4816-bc50-f9a884ecb608 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.714300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.715007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.715353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.715690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.716421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.717856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.777837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c7fb3-94b5-42cd-aa79-475109f11620 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.799817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.810718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 375/749] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:12 2022] POST /identity/v3/services => generated 240 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:12.815747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.818472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:12.819297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:12.819570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:12.819891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:12.834907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:12.836662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:12.848765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:12.854570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:12.855586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:12.857854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:12.857854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:12.857854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:12.857854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 375/750] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:12.857854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:12.977477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de1692-0b7f-4abd-8c6a-90a328f7b546 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:12.989956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09941d3-9b15-47cf-b681-10813b6534d2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.047958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.048955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.049438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.050084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.051199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.051824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.052102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.052366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.061957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.062262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.063937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.064259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:13.077911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:13.078949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 376/751] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:53:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:13.080537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:13.083991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:13.084950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:13.085875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:13.085875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:13.085875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:13.087964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 376/752] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:13.089060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.131027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.175186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.193222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10a47e-8dd6-4f3e-9943-902bd265b0f6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.215480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affae1c3-e8a7-4890-b63f-f1a20c17553d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.230193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.231154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.231670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.232146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.233102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.233608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.235368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.255146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 377/753] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.257310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.262900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.264315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.264801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.265431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.266417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.266978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.268757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.288991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 377/754] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.290342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.371867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.385480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c0a6-a948-42c1-98a3-3a62f5409402 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.404724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1cf83-8f4c-4e04-9d6f-9d6e444b1bda None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.433712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.435346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.436238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.436513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.437346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.438235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.438776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.440359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.444915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.445547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.445830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.446099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.454858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.465175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 378/755] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.465316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.465412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:13.469285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:13.470255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:13.471345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:13.475427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 378/756] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:13.476301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.574966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7a3c1-099b-4a44-92e8-526d4ad6ce8e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.580196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e344b-2c73-428a-8b57-6bd79bdd2e32 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.613275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.614170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.614611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.614970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.615084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.615758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.616106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.616447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.623891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.625679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.627795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.629543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:13.637339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:13.639032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:13.640194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:13.645553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 379/757] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:13.645553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:13.647067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:13.648941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:13.650861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:13.650861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:13.650861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:13.650861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 379/758] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:13.650861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.651854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.696198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.753855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad84d9-338d-4988-8fae-548d9a9e88f4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.759073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5a4eb-d708-46b4-af33-0bc03d0a1ba7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.790986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.791776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.792122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.792463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.793310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.793704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.795280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.798840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.799673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.800152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.800616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.801537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.802020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.803751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.813420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 380/759] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.814561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.818016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 380/760] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.819209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.893406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.926462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a35c56-a215-4383-8152-398d38566d58 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.933531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06f19b-21d2-415d-a2cb-7485e50d7a8a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:13.938750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:13.965007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.965612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.965909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.966756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.967396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.967694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.969068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.971878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:13.972739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.973271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.973731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.974861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=69b58d64a6e04c1487f76e5ea26c74a9) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:13.975377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:13.977056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5281b03-9f0c-4a0d-8133-303b14f333c5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:13.981061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 381/761] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.981976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:13.988181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 381/762] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/role_assignments?user.id=69b58d64a6e04c1487f76e5ea26c74a9&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:13.989431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d214f0-f384-475e-8708-d81413866eb6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:13.989715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d214f0-f384-475e-8708-d81413866eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:13.989988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d214f0-f384-475e-8708-d81413866eb6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:13.991408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 382/763] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:13.991940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d214f0-f384-475e-8708-d81413866eb6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:14.161490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd3c7-6d09-455a-9f6d-c0a87d1f512b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:14.164136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d214f0-f384-475e-8708-d81413866eb6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:14.196083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:14.205766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:14.206499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.206794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.207058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.207714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:14.208543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.209018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.209487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.210412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=bbf97a6003294b5ba411fa9ad2fda18d) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:14.210999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:14.212661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb370e-16fd-44b1-97ce-fffe3e59dfbf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:14.213772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:14.215321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:14.222927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 382/764] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/role_assignments?user.id=bbf97a6003294b5ba411fa9ad2fda18d&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:14.224534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.225054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.225527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:14.231365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:14.232393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:14.233292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:14.233748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 383/765] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:14.234570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:14.271547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:14.273188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:14.331466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:14.392262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876786f-2a29-49d7-9c6d-256390904637 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:14.423039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 383/766] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:14.453972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:14.472733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:14.473978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.474630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.475530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.476857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:14.477367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:14.480287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:14.506134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 384/767] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:14.513300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:14.602318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:14.626453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ccae-849a-40ba-a1a6-5787ae6da1fe None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:14.694784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:14.695420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.695703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.695973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.696556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:14.697923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:14.708657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be91142-3ab8-4dee-b778-8b686fb9641e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:14.715192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:14.716333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:14.717199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:14.722813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 384/768] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Aug 10 22:53:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:14.723681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:14.755397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:14.770083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:14.770776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.771055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.771326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/07cef180b05347ebbcc6f3a6e97dac3a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.776055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=07cef180b05347ebbcc6f3a6e97dac3a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:14.777514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:14.832228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72acd9-0e4e-4535-8949-b9a05a40b83d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:14.865947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 385/769] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:14 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/07cef180b05347ebbcc6f3a6e97dac3a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:53:14.869385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:14.870000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:14.870424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:14.870718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:14.871354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:14.871656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:14.873595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dae3c-e1c8-4f1d-b765-7ed58a04150e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:14.880108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 385/770] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 10 22:53:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:14.885513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.040188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785029-f8c0-438a-adba-6dbed2d5d333 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.049217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:15.071729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4820cf-8a74-408c-90c0-582aa4148029 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.072729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4820cf-8a74-408c-90c0-582aa4148029 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.073565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4820cf-8a74-408c-90c0-582aa4148029 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.076016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 386/771] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:15.080308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.080926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.081202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.081469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.082100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:15.082397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.082975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.083467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.083874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.084134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.089552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 386/772] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 10 22:53:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:15.095802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.109529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:15.111363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:15.199287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 387/773] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:15.222235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.228918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2208276-4b2b-45b4-af4e-5c666bc80530 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.246579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:15.304976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.305502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.305778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.306040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.306757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] RBAC: Adding query filter params (interface=public, service_id=b036e92654294d87af1ead12f8ed8710, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:15.307060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.308538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.330015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 387/774] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/endpoints?service_id=b036e92654294d87af1ead12f8ed8710&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:15.330950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.364910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdb528-de52-492f-84bc-0d49919af321 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.399960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.400607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.400870 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.401141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.401725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.403261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:15.412195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:15.413227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:15.414108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 388/775] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:15.419739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.423033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:15.535202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:15.543815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448d2ea-015e-4fa7-8e2e-6ec18d09ffc2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.552899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3509d-d909-4e8e-a5aa-7c3a6d232fe5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.581252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.581931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.582626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.583170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.584169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] RBAC: Adding query filter params (name=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:15.585541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.587155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.592658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 389/776] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:15.598197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.613334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.614795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.615343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.615824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.616672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.618535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.625791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 388/777] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/services => generated 772 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:15.714827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605226-f041-4335-bfb9-1c36fdf79b24 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.751529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.752247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.752529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.752797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.753431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:15.753731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.755228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.761332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 390/778] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:15.767382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.907020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:15.913002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024acff0-f5e3-4642-9582-5b177acab254 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:15.955301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:15.955967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:15.956255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:15.956511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:15.958686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] RBAC: Adding query filter params (interface=public, service_id=a002cc0f56774af3b971384e2c17a85a, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:15.958686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:15.958980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38609b2d-a02b-4dac-a0a3-8aaa5ca4e26b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:15.965256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 389/779] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/endpoints?service_id=a002cc0f56774af3b971384e2c17a85a&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:15.971149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:15.989640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:16.102366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388443a-c287-459f-81f4-69fd405a10e2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:16.138221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b302a04-e41c-4f0c-97f1-6411f1c90b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:16.138899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b302a04-e41c-4f0c-97f1-6411f1c90b09 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:16.139176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b302a04-e41c-4f0c-97f1-6411f1c90b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:16.139495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b302a04-e41c-4f0c-97f1-6411f1c90b09 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:16.140158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b302a04-e41c-4f0c-97f1-6411f1c90b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:16.141521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b302a04-e41c-4f0c-97f1-6411f1c90b09 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:16.147198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 391/780] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:15 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:16.461106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394016c3-5834-44a4-83b1-7945470395c6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:16.461546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394016c3-5834-44a4-83b1-7945470395c6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:16.461914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394016c3-5834-44a4-83b1-7945470395c6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:16.463905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 390/781] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:16.468739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41209df6-779e-4bdb-b2ea-c60dd2207d2e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:16.469141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41209df6-779e-4bdb-b2ea-c60dd2207d2e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:16.469516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41209df6-779e-4bdb-b2ea-c60dd2207d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:16.489844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-41209df6-779e-4bdb-b2ea-c60dd2207d2e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:16.491277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-41209df6-779e-4bdb-b2ea-c60dd2207d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:16.563536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 392/782] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:16.586431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394016c3-5834-44a4-83b1-7945470395c6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:16.719984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394016c3-5834-44a4-83b1-7945470395c6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:16.766884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6ae87-4e93-4857-98bf-86c52781e067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:16.767506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ae87-4e93-4857-98bf-86c52781e067 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:16.767835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ae87-4e93-4857-98bf-86c52781e067 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:16.768097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ae87-4e93-4857-98bf-86c52781e067 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:16.778563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6ae87-4e93-4857-98bf-86c52781e067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:16.782752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce13324-c2f8-4163-9df7-ef78e4b38260 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:16.783105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce13324-c2f8-4163-9df7-ef78e4b38260 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a8c6ae87-4e93-4857-98bf-86c52781e067 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:16.783387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:16.784283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:16.785146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:16.786348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce13324-c2f8-4163-9df7-ef78e4b38260 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:16.786460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 391/783] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:16.786566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 393/784] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:16.790015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:16.790292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:16.790563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:16.792523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce13324-c2f8-4163-9df7-ef78e4b38260 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:16.811648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:16.822042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:16.824331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:16.864798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:16.957190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 392/785] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:16.974359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce13324-c2f8-4163-9df7-ef78e4b38260 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:16.996802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.019012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d521-b202-4321-a733-865e110b8aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.019743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d521-b202-4321-a733-865e110b8aad None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.020012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d521-b202-4321-a733-865e110b8aad None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.020280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d521-b202-4321-a733-865e110b8aad None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.020946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d521-b202-4321-a733-865e110b8aad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:17.021243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d521-b202-4321-a733-865e110b8aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:17.022820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d521-b202-4321-a733-865e110b8aad None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:17.033579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 394/786] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:17.038989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d521-b202-4321-a733-865e110b8aad None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.112929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:17.150516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d521-b202-4321-a733-865e110b8aad None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.161443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7fc2-5b54-446c-8e9c-ec192ebc8df1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.194087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.194750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.195029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.195298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.214001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:17.215459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:17.229216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.229770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.230045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.230313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:17.244403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:17.245427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:17.246355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:17.246355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:17.246355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:17.251995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 395/787] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:17.253083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:17.258534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:17.259310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:17.260246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:17.261079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 393/788] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:17.270345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.292000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:17.399855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05193e4a-0cd0-4265-a062-1ba4cfd37cb1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.405039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5f685-917a-4789-b760-3a197292a417 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.438327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:17.448719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.449434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.449928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.450383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.451720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:17.452146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:17.453911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464eaca0-7d98-45c9-9e0e-8e6c9b572f55 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:17.455998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.457066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.457535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.457949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.458756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:17.459198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:17.461489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187cb02-a8db-4a7b-ae4e-0dc92c968a8e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:17.463266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 394/789] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:17.465105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1db93-84c5-4b96-b56b-c39c65a1d039 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.465529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1db93-84c5-4b96-b56b-c39c65a1d039 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.465947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1db93-84c5-4b96-b56b-c39c65a1d039 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.470736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 395/790] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:17.471658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1db93-84c5-4b96-b56b-c39c65a1d039 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.485012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 396/791] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:17.486568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.486980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.487352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.506557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:17.507993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:17.556087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:17.586818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 397/792] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:17.588085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.589633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1db93-84c5-4b96-b56b-c39c65a1d039 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.640402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.641068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.641345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.641740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.656540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:17.658314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:17.678005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:17.679088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:17.680612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:17.680612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:17.680612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:17.680612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 396/793] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:17.680612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.737457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f479c-b273-4d75-8837-fd0a6d18e97a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.753558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:17.792084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.792716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.792989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.793288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.804868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:17.807132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:17.812725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c5e3d-3adb-4d18-baeb-af6575130066 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:17.822265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:17.823286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 398/794] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:17.824848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.845828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:17.862131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:17.862800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:17.865352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:17.865352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:17.875095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:17.879782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:17.880716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:17.881568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 397/795] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:17.886791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:17.968827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0940e-9c27-4857-b4b9-c36ce70f1942 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.009731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2879e-0b5e-413c-80f5-6b02ca42f9ce None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.009910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.010649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.011167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.011508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.012296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:18.012597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.014792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.027831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 399/796] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:18.029034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.055725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.056882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.057343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.057985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.059359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:18.060604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.061796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.064132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.088695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 398/797] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 10 22:53:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:18.089974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.137287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08fade-0ad3-4202-b28b-19c819f53c2a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.154548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.175173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.175779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.176058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.176377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.177026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:18.177326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.178694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.188610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 400/798] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:18.189417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.218643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975bb40-99d2-4536-9220-487d323180d3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.255963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.256604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.256886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.257167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.265114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.266467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d98c5d80-e98e-406a-87fa-787c1d41f15f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:18.278193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:18.279420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:18.280404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 399/799] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:18.281109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0df61-7152-4842-81ee-d78c5a05b43f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.281109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0df61-7152-4842-81ee-d78c5a05b43f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.281109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0df61-7152-4842-81ee-d78c5a05b43f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.286408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 400/800] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:18.286408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0df61-7152-4842-81ee-d78c5a05b43f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.295934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.305366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faffdbf-8cdd-4c0b-ac91-bf32e06aa617 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.341557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.342181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.342458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.342777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.353840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.355302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:18.370723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:18.371780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:18.373379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:18.373379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:18.373379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:18.376825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 401/801] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:18.377586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.401791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0df61-7152-4842-81ee-d78c5a05b43f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.405163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.439590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.440189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.440674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.441316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.451831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.453260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d8e023fa-077b-4b6e-83ca-4c858161f2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:18.468150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:18.469126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:18.470062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:18.470062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:18.470062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:18.470062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 401/802] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:18.470838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.471131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.471387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.489926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:18.491146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:18.498646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f58e80-8818-447e-9d06-eca38ee78332 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.535196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.535714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.535995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.536260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.536975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:18.537276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.538705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.556274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 402/803] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:18.557084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.560261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.567904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 402/804] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:18.568904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.663525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.670527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034f59-8a6b-40ac-b9f9-4c004388bbc7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.677981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176a3a-4100-494f-8c47-720d165f5572 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.705122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538667a2-e345-4595-a59f-25158a50032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.705712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538667a2-e345-4595-a59f-25158a50032e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.705988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538667a2-e345-4595-a59f-25158a50032e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.706256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538667a2-e345-4595-a59f-25158a50032e None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.706920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538667a2-e345-4595-a59f-25158a50032e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:18.707220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538667a2-e345-4595-a59f-25158a50032e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.708630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538667a2-e345-4595-a59f-25158a50032e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.712682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.713281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.713557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.713824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.714451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:18.715177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.716115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.719610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 403/805] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:18.720388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538667a2-e345-4595-a59f-25158a50032e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.729356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 403/806] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:18.730171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.783666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.822248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538667a2-e345-4595-a59f-25158a50032e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.829607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d2ca9-41fe-4aa3-8bfa-8e45a3609f76 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.855811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.856943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.857219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.857488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.863048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:18.863855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.863982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.864202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.864771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.866139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.868331 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:18.869739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a56e0935-da89-4c69-b56f-59cc084e2abe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:18.875585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:18.876808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:18.877873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 404/807] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:18.878907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4545cff-cfce-44c6-838e-33664f74726b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:18.878907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4545cff-cfce-44c6-838e-33664f74726b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:18.878907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4545cff-cfce-44c6-838e-33664f74726b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:18.879503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 405/808] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:18.880196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4545cff-cfce-44c6-838e-33664f74726b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:18.891689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:18.892862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:18.893939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:18.893939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:18.893939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:18.893939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 404/809] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:18.893939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:18.894630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:18.984207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4545cff-cfce-44c6-838e-33664f74726b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:18.991766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc308a8-474a-4f39-a0a8-434ab37f2fd5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.006076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.020465 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.021169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.021513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.021913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.025259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.025861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.026136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.026401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.028924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.030464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.034120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.035577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6ff05d69-f92e-4760-9231-5d19670f3403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:19.041424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:19.042505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:19.043606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 406/810] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:19.044619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.044719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.044986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:19.046893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:19.047862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:19.049098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 405/811] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:53:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:19.049796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.064326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:19.065514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:19.126468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.131154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 407/812] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:19.132002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.150728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb371600-e24d-4be9-8dd0-f82439ab7369 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.185307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.185903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.186175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.186445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.187092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:19.187385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.188739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.193011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 406/813] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:19.193756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.234632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.244218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd0768-92b8-446b-b3cb-db2c1656551e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.278663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.279286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.279577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.279829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.280462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:19.280758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.282111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.293107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 408/814] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:19.294376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.294768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156704eb-9fea-4196-9419-f934c70ee413 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.329671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.330264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.330540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.330839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.331517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:19.331817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.333235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.344955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 407/815] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:19.346228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.351850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.399963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029daf02-5469-48fd-bfb0-6439001bdb27 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.446032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.448007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.448693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.449066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.449427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.449589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d017f9-b771-4edc-8891-d57b5b0221b0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.450360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:19.451145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.452933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.469946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 409/816] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:19.470780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.483794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.484423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.484958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.485775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.486360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.487780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:19.496232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:19.498024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:19.498923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 408/817] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:19.500055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.574867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80390f-1eb8-4312-a20b-db97c81909dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.593827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.597160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bcd35-5e11-4cbe-91f9-95c01c1c8cf6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.611476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.612081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.612356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.612625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.613247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:19.613552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.614941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.619986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 410/818] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:19.620734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.636045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.636657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.636929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.637190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.644163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.645597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:19.656865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:19.658222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:19.659504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:19.662628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 409/819] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:19.663355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.688599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.720464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e8741-6ef2-41d9-898e-78dde6f62c69 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.754431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.755118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.755406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.755673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/69b58d64a6e04c1487f76e5ea26c74a9/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.757771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=69b58d64a6e04c1487f76e5ea26c74a9, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.759279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.762835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205b8a-c92e-44ef-beab-39cdbc145472 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:19.801748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:19.802348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:19.802629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:19.802934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:19.859101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 411/820] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:19 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/69b58d64a6e04c1487f76e5ea26c74a9/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:19.860350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:19.878521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:19.880380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:19.905924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.912267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:19.980679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70651a-10b2-4b47-82f9-f28319972fb6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.003119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 410/821] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:53:20.006947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.021079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.021697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.021973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.022243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.022912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:20.023256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.024627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.030810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 412/822] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:20.031612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.122225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ff5c-a774-48e5-bb29-1491b8ac1d71 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.144761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4061-7f6f-4404-92f1-a3441f52d2a4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.146855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.154383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.159555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.160228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.160504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.160769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.161429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.162892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.181249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.181873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.182129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.182397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.184243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:20.184243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.184859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.196824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 413/823] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:20.197630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.286370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 411/824] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:20.295490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996615df-3e34-4669-9f1f-26a7eabe69c1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.295729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.359858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.360771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.361195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.361715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.378303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.380060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:20.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:20.404387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:20.405516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:20.405516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:20.405516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:20.408788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 414/825] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:20.410010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.437672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.457643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95cc4f-1316-4073-a11e-415ed3064479 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.499857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.500808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.501266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.501678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.502477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] RBAC: Adding query filter params (type=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:20.502988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.504597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.510233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 412/826] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:20.511306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.558881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.566475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2be66-f07a-4a6a-833a-b25c464cd17b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.613092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.614033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.614325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.614602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/bbf97a6003294b5ba411fa9ad2fda18d/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.620429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de788-aa18-4199-80d4-107c2ccbdd2a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.635085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=bbf97a6003294b5ba411fa9ad2fda18d, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.637193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.657568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.658170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.658443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.658744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.659381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:20.659681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.661038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.673884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 413/827] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:20.674585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.739076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 415/828] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:20 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/bbf97a6003294b5ba411fa9ad2fda18d/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:20.740389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:20.816568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a8b97-c183-44cb-a268-1101c90914e3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.819020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.861110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.861966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.863025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.863443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.864437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.866350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:20.893111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.937862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080c660-fe22-4b21-bffb-5a73072af5e1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:20.963832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 414/829] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:20 2022] POST /identity/v3/endpoints => generated 337 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:20.987226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:20.988031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:20.988421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:20.988839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:20.996863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:20.999379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:21.013451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:21.014415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:21.015466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 416/830] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:21.020473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:21.185601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21fe14-2af4-4a8e-af2d-086934621033 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:21.222556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:21.223570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:21.224065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:21.224560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:21.225448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:21.225955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:21.227627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7ae1f-4f7b-4ba5-b7a5-98bb0e5a60f1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:21.241769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 415/831] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:21.249896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:21.369865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35533eb0-5023-434c-a625-87730d5784d6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:21.391854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:21.408457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:21.412069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:21.412069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:21.412069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:21.412069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=07cef180b05347ebbcc6f3a6e97dac3a) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:21.412069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:21.412069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1c03-e19d-4445-ae6c-f8be7b5378f6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:21.418766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 417/832] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:21 2022] GET /identity/v3/role_assignments?user.id=07cef180b05347ebbcc6f3a6e97dac3a&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:22.024345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4603acd-1aab-46a9-b60a-6ad793913246 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.024859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4603acd-1aab-46a9-b60a-6ad793913246 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.025262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4603acd-1aab-46a9-b60a-6ad793913246 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.026961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 416/833] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:22.032852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.033306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.033591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.052940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:22.054293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:22.125402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 418/834] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:22.148826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4603acd-1aab-46a9-b60a-6ad793913246 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:22.168571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:22.255251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4603acd-1aab-46a9-b60a-6ad793913246 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:22.292305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:22.292849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.293127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.293397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.304576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:22.309246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:22.310716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:22.311591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 417/835] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:22.317070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:22.417273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8a28e-b0af-4cad-beba-44f15304c9aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:22.439323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:22.454337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:22.454973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.455253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.457282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.457282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:22.457282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:22.457951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cfb106-6113-4c27-95bb-95da77604012 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:22.466882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 419/836] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:22.472098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:22.515417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609217da-a56c-450c-ad66-ae73e2be1287 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.515731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609217da-a56c-450c-ad66-ae73e2be1287 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.515963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609217da-a56c-450c-ad66-ae73e2be1287 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.517592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 420/837] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:22.522270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.522547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.522847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.541783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:22.543158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:22.563154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:22.585559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf038-1007-4606-883e-38a78dc64cf4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:22.623299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:22.624413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.624781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.625145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.626005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 421/838] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:22.643679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:22.646679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:22.666150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:22.674698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:22.676674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:22.678595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:22.678595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:22.678595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:22.686680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 418/839] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:22.686680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:22.862764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc7035-c0b6-4407-b665-79ec8a6ab979 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:22.878966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:22.891719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1639-ec27-4e3e-b4eb-f8f24fe43a09 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:22.932218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:22.933106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.934084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.935687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.936679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:22.939312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:22.952358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:22.955834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:22.956843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:22.957689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 422/840] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:22.964513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:22.974605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:22.974605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:22.974605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:22.974605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:22.975147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:22.975147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:22.975147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.006681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 419/841] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:23.008793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.149438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da001c-0419-434a-9f59-3f97485cedc7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.170042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07301-4e26-4787-ab25-e1b0a437cdda None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.186054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.186784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.187058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.187323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.187943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:23.188234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:23.189707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43c765-91e8-458b-ad43-edbada3d3469 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.195362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 423/842] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:23.196811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f3a57-bd9d-4eaf-8209-31bc8030b819 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.197088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f3a57-bd9d-4eaf-8209-31bc8030b819 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.197361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f3a57-bd9d-4eaf-8209-31bc8030b819 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.203682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 424/843] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:23.203682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f3a57-bd9d-4eaf-8209-31bc8030b819 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.211355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.212204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.212692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.212963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.220763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:23.222188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-eaae09e3-b079-4244-b8bc-10cba4316296 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:23.235104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:23.236260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:23.237241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 420/844] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:23.237860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.237860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.237987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.244640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:23.257087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:23.258364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:23.310038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:23.315335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f3a57-bd9d-4eaf-8209-31bc8030b819 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.342681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 421/845] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:23.342681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.364431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.365034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.365305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.365567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.366182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:23.366474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:23.367854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.372575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 425/846] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:23.373365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.456143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cc7f4-d708-424d-837a-7b5ec6cdb1e3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.493401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.494013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.494286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.494549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.495325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:23.495625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:23.497643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.511256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:23.517819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 422/847] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:23.518917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ca1-65a2-49ac-a31e-14d0e5141d33 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.523566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.563679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b63f80-ce61-479d-8d9e-e8e947f8197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.564306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63f80-ce61-479d-8d9e-e8e947f8197b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.564580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63f80-ce61-479d-8d9e-e8e947f8197b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.564844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63f80-ce61-479d-8d9e-e8e947f8197b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.571584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:23.579978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b63f80-ce61-479d-8d9e-e8e947f8197b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-27b63f80-ce61-479d-8d9e-e8e947f8197b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:23.585911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:23.586906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:23.587784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 426/848] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:23.589001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab741d36-394e-460a-afb1-6dea321bc7dc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.589100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab741d36-394e-460a-afb1-6dea321bc7dc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.589362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab741d36-394e-460a-afb1-6dea321bc7dc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.594923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 427/849] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:23.595731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab741d36-394e-460a-afb1-6dea321bc7dc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.641734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fbf10-ca58-4d94-b0e4-c46f791ea26d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.677826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.678340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.678613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.679115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.703167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab741d36-394e-460a-afb1-6dea321bc7dc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.737699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:23.738313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.738599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.738903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.739617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:23.739925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:23.741295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a62010-0c2f-4d01-9ea0-8f815f3010d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 428/850] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:23.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:23.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:23.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:23.758523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:23.760013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:23.770935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:23.772128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:23.786845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:23.800488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:23.842383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 429/851] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:23.843073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.883304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 423/852] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:53:23.883697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:23.961920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c935e92-f5f6-466d-a45d-740b563f9ea0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.999386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f864e-ac3b-489a-a30c-63d9da80ba42 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:23.999617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.000204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.000479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.000751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.011123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.012593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:24.027058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:24.033069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:24.034080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:24.034080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:24.034080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:24.034080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 430/853] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:24.034080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.034080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.044329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.044959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.045242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.045514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.050558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.050558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:24.057705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:24.058718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:24.060590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:24.062973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 424/854] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Aug 10 22:53:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:24.064276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.067348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.137797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d0fb1-8fd4-4e0e-bb2e-758cb2b50f22 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.168653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fb2ed-aa15-466b-9e63-aeae765d29e7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.175035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.176283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.176729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.177148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.188808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.190228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:24.204668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:24.205600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:24.206499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:24.206499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:24.206499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:24.206499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 431/855] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:24.206499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.209774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.210378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.210674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.210951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.211585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:24.211885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.213291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.225510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 425/856] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:24.226300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.273735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.292347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.319371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc49f8-edab-4817-8a69-9ebc130ba11f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.330618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08731f-bd1b-4a19-b9f2-b739493bbeec None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.354530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.355186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.355467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.355738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.356909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:24.357210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.358562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.367432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 432/857] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:24.367432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.374638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.375332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.375611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.375885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.376582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:24.376888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.378245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.395006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 426/858] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:24.395006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.480832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72c08-9584-4088-a1ee-a523438f6995 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.499781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.511845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33397d2-3954-4bda-b236-27d62bfddc6c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.520600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.521243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.521521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.522165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.530879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.532313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.536713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:24.543677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:24.544669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:24.545642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:24.549494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 433/859] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:24.550326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.551230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.551832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.552110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.552431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.553175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:24.553477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.554920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.559693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 427/860] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:24.560526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.675280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d45d4a-7fa2-4c07-9e4e-b8a3e8cf0bf4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.711065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae58c5-9556-4aff-9aa7-2bec6dab7450 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.724752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.725384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.725665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.725930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.738494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.739903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.750547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.751254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.751575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.751842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.752557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:24.752854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:24.752952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:24.753878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:24.754879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:24.755330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.755403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 434/861] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:24.762486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.780408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 428/862] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:24.786079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.810743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.849913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:24.884121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722410a-c958-47e0-8a3b-802b8b3d7f61 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.908384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8028ae-8fb0-46ba-beb9-03c28afba278 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:24.932485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.933250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.933610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.933990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:24.934838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:24.935239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:24.936796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:24.954905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 435/863] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:24.956039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:24.957403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:24.958188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:24.958605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:24.959042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.046647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.048813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.083505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:25.089488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936551-7b67-4463-815a-d3aed32f7b15 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:25.105205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:25.133405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:25.134005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.134280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.134548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.135295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=69b58d64a6e04c1487f76e5ea26c74a9) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:25.135596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.137014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.151091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 436/864] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/role_assignments?user.id=69b58d64a6e04c1487f76e5ea26c74a9&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:25.169501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 429/865] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:53:25.175018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:25.291865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144fb0-2b70-4d1c-8f64-4b0a0a1296f0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:25.329263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:25.329804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.330080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.330343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.341030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.342537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-17871ecb-057a-465f-b224-f0a6dacfa0d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:25.358380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:25.359418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:25.360335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:25.360335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:25.360335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:25.360335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 437/866] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:25.366748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:25.435282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df04cc-aa78-4d3c-b97a-087e152f2265 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.435564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df04cc-aa78-4d3c-b97a-087e152f2265 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.435833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df04cc-aa78-4d3c-b97a-087e152f2265 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.437412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 438/867] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:25.442510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.442847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.443115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.462043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:25.463719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:25.475029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:25.483767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84563965-5244-44bf-92ec-04581dcc5923 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:25.519727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:25.520327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.520613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.521125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.521508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:25.521857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.523269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.539343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 430/868] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:25.540765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:25.553174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 439/869] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:25.563014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:25.576188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:25.668459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a355e-75a7-4e56-9ca1-ccecd9831c10 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:25.684414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5143b5-9b76-49f7-a61c-4c8eea912dbc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:25.706409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:25.707128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.707416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.707688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.714769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.716742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:25.728583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:25.729522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:25.730455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:25.738470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:25.738634 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 431/870] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:25.738757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.738757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.738757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.738962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:25.739046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.739046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.803907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:25.820461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 440/871] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:25 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:25.850773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87f6d-775a-4b77-9aa2-65d1a01c56ec None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:25.898031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:25.898726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:25.899011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:25.899283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:25.899977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:25.900339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:25.901869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838047c-3f69-4842-bd96-494625b01585 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:25.916935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 432/872] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:25.929803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:25.931800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:26.050371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d911f1c-31ed-4d67-9c09-be8736a45fcf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:26.084359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:26.085022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:26.085303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:26.085572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:26.086355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=bbf97a6003294b5ba411fa9ad2fda18d) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:26.086682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:26.088187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe5ee8-b5fa-4b8e-a4dd-a56d7b195fad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:26.095004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 441/873] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:25 2022] GET /identity/v3/role_assignments?user.id=bbf97a6003294b5ba411fa9ad2fda18d&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:26.803994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6352e4b-1951-4d6e-b970-484a39b84234 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:26.804773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6352e4b-1951-4d6e-b970-484a39b84234 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:26.804773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6352e4b-1951-4d6e-b970-484a39b84234 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:26.807434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 433/874] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:26.816142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9b31d-ae1a-478e-9d7c-37a54aa86e79 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:26.816804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9b31d-ae1a-478e-9d7c-37a54aa86e79 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:26.817124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9b31d-ae1a-478e-9d7c-37a54aa86e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:26.845542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a7d9b31d-ae1a-478e-9d7c-37a54aa86e79 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:26.847593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a7d9b31d-ae1a-478e-9d7c-37a54aa86e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:26.959514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:26.962335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 442/875] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:26 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:26.985869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6352e4b-1951-4d6e-b970-484a39b84234 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:27.093173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6352e4b-1951-4d6e-b970-484a39b84234 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:27.114258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:27.128644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b6d3-1bf2-4979-bb53-9806e941c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:27.130932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b6d3-1bf2-4979-bb53-9806e941c197 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.130932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b6d3-1bf2-4979-bb53-9806e941c197 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.130932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b6d3-1bf2-4979-bb53-9806e941c197 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.130932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b6d3-1bf2-4979-bb53-9806e941c197 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:27.131829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b6d3-1bf2-4979-bb53-9806e941c197 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:27.297830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 434/876] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:26 2022] POST /identity/v3/services => generated 269 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:27.332001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cf0ec-d9c3-40c4-9d81-8683aae5e6fd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.332324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cf0ec-d9c3-40c4-9d81-8683aae5e6fd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.332552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cf0ec-d9c3-40c4-9d81-8683aae5e6fd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.334020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 443/877] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:27.340021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.340691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.341177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.361197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:27.362928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:27.450764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 435/878] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:27.470549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cf0ec-d9c3-40c4-9d81-8683aae5e6fd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:27.581089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cf0ec-d9c3-40c4-9d81-8683aae5e6fd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:27.616484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:27.617367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.617847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.618266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.619060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:27.620571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:27.630550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:27.631807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:27.632940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:27.638784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 444/879] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:27.647752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:27.749270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0887c5-c2b3-4ee3-ae0e-2c1246621917 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:27.756833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:27.783719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:27.784320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.784594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.784860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.785482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:27.785779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:27.787166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce48e0a-5330-4465-a017-ec35addde623 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:27.792154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 436/880] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:27.798061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:27.865201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:27.906388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373fab-8004-4f1a-b860-aa62f47e5c0c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:27.928771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28962f-5276-45cf-9fcc-e91f4ca4014e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.929748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28962f-5276-45cf-9fcc-e91f4ca4014e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.930213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28962f-5276-45cf-9fcc-e91f4ca4014e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.932102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 437/881] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:27.937806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.938185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.938463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.942839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:27.943439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:27.943711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:27.943976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:27.944689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:27.944983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:27.946384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:27.951387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 445/882] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:27.963889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:27.965316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:28.061249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 438/883] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:28.083598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.195099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dec9d-2eee-4491-aaff-53a5e2fdc333 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:28.221383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:28.230190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:28.231401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.231685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.231952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.232530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:28.233881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:28.242887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:28.243900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:28.244759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 446/884] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:28.252911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.365000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:28.368216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0ff4e-89f5-4178-85a7-83eb322179fd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:28.403479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:28.404150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.404431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.404705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.405330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:28.405630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:28.407205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c3e76-ebf2-43f9-9105-9828b7dab112 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:28.413696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 439/885] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:28.419895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.520123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5d4a-bea8-4516-b0db-5c68c660e0b6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.520304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5d4a-bea8-4516-b0db-5c68c660e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.520578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5d4a-bea8-4516-b0db-5c68c660e0b6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.522054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 440/886] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:28.526932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.527213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.527489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.534697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e55418-aa74-434e-9672-6caca7682e79 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:28.546886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:28.548103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:28.583804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:28.584417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.584696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.584966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.585594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:28.585897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:28.587309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:28.593297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 447/887] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:28.598606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.645244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 441/888] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:28.670143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.674283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:28.720744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5258e-b625-46f1-b84d-5a064788620c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:28.755685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946880-5d28-4199-9d69-736013bc680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:28.756282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946880-5d28-4199-9d69-736013bc680a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.756558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946880-5d28-4199-9d69-736013bc680a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.756885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946880-5d28-4199-9d69-736013bc680a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.757519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82946880-5d28-4199-9d69-736013bc680a None admin] RBAC: Adding query filter params (interface=public, service_id=2f605ff66799438aa26db93a94d9a80e, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:28.757821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82946880-5d28-4199-9d69-736013bc680a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:28.759214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82946880-5d28-4199-9d69-736013bc680a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:28.769473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 448/889] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/endpoints?service_id=2f605ff66799438aa26db93a94d9a80e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:28.770379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946880-5d28-4199-9d69-736013bc680a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.771145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:28.777670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cc57b-8629-4fc8-ad4a-9dad60921cca None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:28.817974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:28.818629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.818935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.819203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.830933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:28.836186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:28.837137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:28.837988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:28.842829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 442/890] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:28.844273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:28.872667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946880-5d28-4199-9d69-736013bc680a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:28.891175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:28.909236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c6cda-7232-472c-b462-3b949181227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:28.909838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c6cda-7232-472c-b462-3b949181227f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:28.910114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c6cda-7232-472c-b462-3b949181227f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:28.910385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c6cda-7232-472c-b462-3b949181227f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:28.911167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c6cda-7232-472c-b462-3b949181227f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:28.913106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c6cda-7232-472c-b462-3b949181227f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:28.922703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 449/891] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:53:28.966128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663b891-6a37-4e35-992a-aecaf3b0faa1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.005355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.006320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.006846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.007355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.008233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:29.008789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:29.010381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:29.021056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 443/892] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:53:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:29.027656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c6cda-7232-472c-b462-3b949181227f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.140167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c6cda-7232-472c-b462-3b949181227f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.177405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.178389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.178917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.179388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.192343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:29.194075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:29.209406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:29.210495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:29.211488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:29.211488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:29.211488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:29.213298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 450/893] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:29.219325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.239315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:29.323854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b645a-1421-4c30-9927-b26fbe85abd8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.358674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.359277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.359552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.359821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.360457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:29.360752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:29.362169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b189b-7ccd-40ea-8579-58b1481873a9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:29.375387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 444/894] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:29.381051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.417357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:29.487953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c7fa2-2b31-4b9e-8ef8-edc4174b6e60 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.492030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2171f-97ec-4194-8316-be1a75e74f67 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.492030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2171f-97ec-4194-8316-be1a75e74f67 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.492030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2171f-97ec-4194-8316-be1a75e74f67 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.494131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 445/895] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:29.500089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.500496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.500871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.522694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:29.524043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:29.526367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.527197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.527471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.528281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.536011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:29.537502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:29.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:29.550037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:29.551021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:29.554794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 451/896] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:29.556267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.619062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 446/897] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:29.642967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.651723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:29.665004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:29.671560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e3d4-2b90-40a4-84dc-073270cc26a4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.709014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.710166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.710892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.711547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.713175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:29.714207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:29.716814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:29.743798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 452/898] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:29.745731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.757959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c044d7a-e6e8-4f5b-a831-e4da608b0232 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.803938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.804790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.805077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.805406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.806042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:29.808202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:29.819880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:29.820884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:29.821825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 447/899] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:29.830667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:29.879195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03012f-36ad-4e38-ba7a-8ee09a8632da None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:29.920413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:29.928411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02545b-cd84-4de9-9e82-40113be2e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:29.929531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02545b-cd84-4de9-9e82-40113be2e5bc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:29.929873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02545b-cd84-4de9-9e82-40113be2e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:29.930214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02545b-cd84-4de9-9e82-40113be2e5bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:29.958704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:29.960171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be088598-e6cd-4110-9b68-b223d10d8ec7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.006444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.007889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.008400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.008889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.009782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:30.010287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.012946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.014045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02545b-cd84-4de9-9e82-40113be2e5bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.016094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02545b-cd84-4de9-9e82-40113be2e5bc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.027030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 448/900] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:30.029821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.136705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b7906-1722-45c0-aeae-e29432149049 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.146712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 453/901] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:29 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:53:30.153121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b91ba-4ca5-404d-9874-01f1586fad84 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.153467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b91ba-4ca5-404d-9874-01f1586fad84 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.153830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b91ba-4ca5-404d-9874-01f1586fad84 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.155845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 454/902] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:30.157086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b91ba-4ca5-404d-9874-01f1586fad84 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.176026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.176876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.177302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.177764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.178387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:30.178708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.180072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342809c-c328-4eee-8341-e9e236867614 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.181482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:30.187842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 449/903] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:30.191905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.192176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.192446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.210752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:30.211577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:30.221287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:30.272424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b91ba-4ca5-404d-9874-01f1586fad84 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.286583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 450/904] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:30.287450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.320612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.321339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.321676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.322013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.332602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.334011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:30.348168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:30.349339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:30.350241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:30.350241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:30.350241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:30.350241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 455/905] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:30.350467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.403478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa55d-cb94-43ff-aac4-9f4e7f2196bc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.439036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:30.447508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.448132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.448406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.448671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.449354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] RBAC: Adding query filter params (interface=public, service_id=e98efd4d45a940acbf809fc51f728b57, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:30.449685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.451068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.453877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90f6be-7932-418f-be37-a3ab5969f931 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.457367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:30.462956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 451/906] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/endpoints?service_id=e98efd4d45a940acbf809fc51f728b57&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:30.462956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.492507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.493101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.493387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.493640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.494269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.495667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.569803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f6af2-c355-4b26-9898-4260060c8ab8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.586909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 456/907] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:30 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:30.586909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.643766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.644514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.644792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.645060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.646050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:30.646352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.648921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.668238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 452/908] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:30.673657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.748306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:30.764607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f23308-99ca-473b-aaeb-6d364fc65db5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.803267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.803877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.804146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.804455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.805102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.806469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.818759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 457/909] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/services => generated 1497 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:53:30.818947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03f367-30d8-44b9-97c8-c2fd360e7e8e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:30.865771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:30.866415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:30.866725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:30.867004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:30.876443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:30.879266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-1d8fd26d-1209-4dbd-b4c0-d73cfbafb17a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:30.896559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:30.897553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:30.899395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 453/910] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:30.909755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:30.958971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:31.041186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272881d2-64c1-485a-a931-406b5e7fa7d3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:31.080051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:31.080051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.080051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.080051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.081050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:31.081050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:31.081190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9e675-bed1-4030-87fc-05b3a3634f08 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.082761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9e675-bed1-4030-87fc-05b3a3634f08 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.082761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9e675-bed1-4030-87fc-05b3a3634f08 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.082929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:31.083155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 454/911] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:31.091248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.091248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.091248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.098057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 458/912] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:31.103855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:31.109277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:31.110017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:31.198777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 455/913] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:31.216064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:31.217445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:31.232211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295b9e5-78c2-4387-94a4-bbe5c477b76d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:31.269888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:31.284762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:31.285793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.286507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.286977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.288016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] RBAC: Adding query filter params (role.id=2ca8c476a62e417c86da67ab92dbf61c, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=bbf97a6003294b5ba411fa9ad2fda18d) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:31.288478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:31.290877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:31.300962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 459/914] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:31 2022] GET /identity/v3/role_assignments?user.id=bbf97a6003294b5ba411fa9ad2fda18d&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=2ca8c476a62e417c86da67ab92dbf61c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:31.352647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43779-90ed-4029-b546-a68d25e1e5dd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:31.395827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:31.396473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.396744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.397007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.397582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:31.399097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:31.413131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:31.414461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:53:31.415561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:31.418145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 456/915] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:31.425429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:31.544219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e04c7e-e6ef-4689-84a8-3fb226a60c59 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:31.581177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:31.582056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.582478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.582944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.583707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:31.584145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:31.585654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:31.591647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 460/916] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:31.602733 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:31.728823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62981a-75bf-4118-b038-f41ed2c7768c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:31.731355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:31.765400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:31.766005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.766278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.766545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.767203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:31.767501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:31.768860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e49fa8-af38-4db5-ad59-32c581e4d4cc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:31.774164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 457/917] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 10 22:53:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:31.780620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:31.799177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:31.883198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7195-9834-47eb-b5be-7ae3cc0fa1c1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:31.921459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d0712-f4a9-4c70-90e7-c62eeec2f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:31.922871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0712-f4a9-4c70-90e7-c62eeec2f420 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:31.923376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0712-f4a9-4c70-90e7-c62eeec2f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:31.923871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0712-f4a9-4c70-90e7-c62eeec2f420 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:31.924749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0712-f4a9-4c70-90e7-c62eeec2f420 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:31.926344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0712-f4a9-4c70-90e7-c62eeec2f420 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:32.076777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 461/918] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:32.829817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dc462-424e-425b-a8e7-0f366fcc782b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:32.830308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dc462-424e-425b-a8e7-0f366fcc782b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:32.830308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dc462-424e-425b-a8e7-0f366fcc782b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:32.831924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 458/919] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:32.837387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191bc8b-7af2-4b49-b48b-3853d7b6fb07 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:32.837674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191bc8b-7af2-4b49-b48b-3853d7b6fb07 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:32.837950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191bc8b-7af2-4b49-b48b-3853d7b6fb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:32.856338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f191bc8b-7af2-4b49-b48b-3853d7b6fb07 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:32.857517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f191bc8b-7af2-4b49-b48b-3853d7b6fb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:32.927690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:32.935615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 462/920] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:32.958789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dc462-424e-425b-a8e7-0f366fcc782b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.043551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25d01-054e-44a6-8e64-754605b0853c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.043909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25d01-054e-44a6-8e64-754605b0853c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.044104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25d01-054e-44a6-8e64-754605b0853c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.045704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 463/921] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:33.050605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.051038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.051412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.066648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dc462-424e-425b-a8e7-0f366fcc782b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.070876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:33.072304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:33.102421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.102991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.103280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.103539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.104116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.105563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:33.114592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:33.115749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:33.116603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 459/922] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:33.122399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.159018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:33.160067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 464/923] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:33.174638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.228019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e0c4e-3e82-4dd2-83f8-8993fa1d5269 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.275357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.275980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.276255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.276526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.277167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:33.277503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.278991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.289549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 460/924] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:33.290697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.315529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ac682-2885-447a-bd8c-075caafd6bde None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.350440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:33.365977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.365977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.365977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.365977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.366450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.366450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e3d0c288-4d78-44d7-8f5b-55783492c8ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:33.372291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:33.373258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:53:33.374409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:33.378511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 465/925] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:33.379907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c958be4-1787-4b21-9a30-ff0319b01609 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.380191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c958be4-1787-4b21-9a30-ff0319b01609 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.380460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c958be4-1787-4b21-9a30-ff0319b01609 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.382803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 466/926] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:33.382803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c958be4-1787-4b21-9a30-ff0319b01609 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.396246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:33.402910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea9ee2-b5c6-4fc0-937c-9473bc33bfec None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.447089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.447737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.448043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.448306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.448889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:33.449184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.450625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c82f18-8599-4129-b1f4-8cd46d47d3ad None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.461042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 461/927] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:33.461042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.461042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.461042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.476292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:33.477269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:33.508015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c958be4-1787-4b21-9a30-ff0319b01609 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.545486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.546676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.546964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.547237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.547858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:33.548161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.549529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.549997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 462/928] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:33.551945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.558798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 467/929] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:33.560598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.624038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:33.656982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:33.667688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46003f-ee41-4f6b-a6cb-f6ca70c3fe02 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.670222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea3cf5-a3f6-417b-bcbd-6dfbcbb47955 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.703228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.703831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.704103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.704272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.704386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.704827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.705011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:33.705132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.705284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.705379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.706027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] RBAC: Adding query filter params (interface=public, service_id=f5ffab5ee74c4a8992c35771bb2fe7ee, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:33.706329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.706593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.707705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.711870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 468/930] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:33.712664 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.713360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 463/931] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/endpoints?service_id=f5ffab5ee74c4a8992c35771bb2fe7ee&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:33.722999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.817945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20146e6-a688-49f8-aa76-a8a0b0672142 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.826059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76b423-cd37-4285-8399-3961a5e50265 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:33.839219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:33.853610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.854233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.854567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.854813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.876660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:33.876660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:33.876660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:33.876660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:33.877458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:33.877458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:33.877615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:33.878589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:33.879780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 469/932] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:33.880779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:33.977334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53088a7d-0841-4865-ae8b-924d75263e7e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.000581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 464/933] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:34.001611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.023203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.034170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.035128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.036345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.036619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.037299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.041433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.050978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 470/934] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:33 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:53:34.147355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c802-e084-4e2d-a1c5-80350b83a3b8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.189154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.190247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.190771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.191250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.192265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:34.192863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.194605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6e315-73ca-4c5e-9520-f9f7eafec7cc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.205777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 465/935] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:34.211380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.222116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee5dc6-5111-47b6-86bf-bdd0c51fff40 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.222781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee5dc6-5111-47b6-86bf-bdd0c51fff40 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.223880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee5dc6-5111-47b6-86bf-bdd0c51fff40 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.226039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 466/936] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:34.230944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.231220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.231495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.253981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:34.255584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:34.265715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.331672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d758499-2fe7-4165-9877-ffdf8480364b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.342682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.349134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 467/937] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:34.371782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.372514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.372783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.372903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.373235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.391583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.394027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:34.414374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:34.416494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:34.418213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:34.418213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:34.418213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:34.418213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 471/938] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:34.428446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.481834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100761ca-af5d-43fd-9b9c-58535b09c99c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.518827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.519528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.519878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.520213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.520944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.522454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.525164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING keystone.server.flask.common [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 22:53:34.531018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.537254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d853ca1-dd92-43c5-a454-0ea2b7114f21 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.565925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.sql.core [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 22:53:34.565925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 22:53:34.565925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [parameters: {'id': 'ed88d47d43a748198e713b26e451149b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 22:53:34.565925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81639) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed88d47d43a748198e713b26e451149b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed88d47d43a748198e713b26e451149b. Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:53:34.573596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result.read() Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 22:53:34.575037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application yield resource Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise self.value Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application session.commit() Aug 10 22:53:34.576030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.session.flush() Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application compat.raise_( Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 22:53:34.576937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 22:53:34.577887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application result.read() Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:53:34.579669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [parameters: {'id': 'ed88d47d43a748198e713b26e451149b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.581205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:34.582216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed88d47d43a748198e713b26e451149b. Aug 10 22:53:34.583407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.584030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.584030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.584030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.584030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.584366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:34.584449 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 468/939] 10.222.0.78 () {60 vars in 1140 bytes} [Wed Aug 10 22:53:34 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:34.584531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.584531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.590715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.594980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 472/940] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:34.604507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.650817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.698709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8e16-edda-4594-ab1c-d48c620da795 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.717213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280a3da-a405-4ad2-8acf-02c777cb16e7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.737490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.738153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.738554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.738872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.746104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.748313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.758049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.758792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.759132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.759459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:34.760477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:34.762301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:34.764490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:34.770706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 469/941] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:34.770706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.771093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.771093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:34.782197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:34.784400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:34.786682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:34.794699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 473/942] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:34.794699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:34.807091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.874429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624d237-50e5-4816-9cdc-c5288cb1d4b1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.891187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:34.904040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b856b-ba43-4157-a553-bcc3a48ba97d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:34.911046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.911704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.912083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.915250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.915250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:34.915250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.915250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.940832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:34.941665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:34.941939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:34.942204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:34.942969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:34.943145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 470/943] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:34.943314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:34.944851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:34.968947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 474/944] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:34.974735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:35.090287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a38cb-e3e7-4bdd-a409-29095b76c90b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:35.127874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:35.128498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:35.128778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:35.129050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:35.140730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:35.142185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:35.156752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:35.158930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:35.160189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:35.160189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:35.160189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:35.160189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 471/945] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:35.163697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:35.239204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:35.271795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34832423-2991-4d48-9900-6b039a2d6e4d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:35.309630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:35.310231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:35.310501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:35.310796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:35.311433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:35.311727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:35.313067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:35.333386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 475/946] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:35.338690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:35.370336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:35.451320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278e3bb-f4ff-428b-bd50-96a955f89bc5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:35.488682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:35.489659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:35.490128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:35.490569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:35.497951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:35.499861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:35.510956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:35.511906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:35.512819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:35.518760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 472/947] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:35.520633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:35.630711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc62a1-5151-40c8-8288-c5623e64b601 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:35.660355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:35.666755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:35.673694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:35.673694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:35.673694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:35.673694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:35.673694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:35.673694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d1ae-aa62-4eca-b5f2-246a14c1b7d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:35.683661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 476/948] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:35.689488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:35.802826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:35.818150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc22fe0-a8a2-403b-8b9e-73c1bd18344c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:35.853585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a50ea2-8f0f-44d1-8ce0-19b00e4501b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:35.854290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a50ea2-8f0f-44d1-8ce0-19b00e4501b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:35.854567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a50ea2-8f0f-44d1-8ce0-19b00e4501b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:35.854868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a50ea2-8f0f-44d1-8ce0-19b00e4501b1 None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/bbf97a6003294b5ba411fa9ad2fda18d/roles/2ca8c476a62e417c86da67ab92dbf61c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:35.857116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a50ea2-8f0f-44d1-8ce0-19b00e4501b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=bbf97a6003294b5ba411fa9ad2fda18d, role_id=2ca8c476a62e417c86da67ab92dbf61c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:35.858535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a50ea2-8f0f-44d1-8ce0-19b00e4501b1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:35.967032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 473/949] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:35 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/bbf97a6003294b5ba411fa9ad2fda18d/roles/2ca8c476a62e417c86da67ab92dbf61c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:36.167546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd190bd3-b85d-4734-b6e6-0e6709426066 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.168583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd190bd3-b85d-4734-b6e6-0e6709426066 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.168661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd190bd3-b85d-4734-b6e6-0e6709426066 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.170242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 477/950] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:36.175738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.176033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.176309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.195896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:36.197418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:36.232652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b5210-721c-4dbb-a084-49e341320588 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.232966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b5210-721c-4dbb-a084-49e341320588 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.233189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b5210-721c-4dbb-a084-49e341320588 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.234796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 478/951] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:36.239840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.240129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.240382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.263796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:36.265846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:36.272384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 474/952] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:36.295534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.340972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 479/953] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:36.367617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.399577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:36.438879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec855e-cd46-497f-a7b6-3d3953d7624c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.475203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.475882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.476160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.476431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.477017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.479487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:36.489021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:36.490154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:36.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:36.498843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 475/954] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:36.498843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.499184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da39e7-7119-4008-b548-1d38fbce7518 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.523531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:36.539015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.539677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.539955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.540226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] PATH_INFO: `/v3/services/compute` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.540809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.542795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:36.552286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:36.553554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:36.554634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 480/955] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:36.562015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.638370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae741d8-48d8-40fa-93d2-140064f7625f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.675616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566816c-3ff4-4c0f-b772-fd3e03310bff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.682396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.683054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.683452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.684637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.685081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:36.685451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.687232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.696982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 476/956] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:36.698335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.708307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:36.718453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.719518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.726212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.726212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.726212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:36.726212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.726212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.731086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 481/957] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:36.737421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.806081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:36.820612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b88e8f-5000-42be-960b-2a949c6096f0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.856143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.856872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.857203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.857547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.858252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:36.858688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.860451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.871518 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 477/958] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:36.872957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.875451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b27d6-5b76-4d2c-8179-4de53263066d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.915823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.916830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.917333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.917801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.919274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:36.919775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.921440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b56ced-868a-42f4-abd7-fc4051efeec5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.935270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 482/959] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:36.950956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:36.987022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55b16b-ebb3-4f9d-9afb-035b6f35e056 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:37.025593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93e475-5cab-4115-bccd-1d73267c9944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:37.026209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93e475-5cab-4115-bccd-1d73267c9944 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:37.026488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93e475-5cab-4115-bccd-1d73267c9944 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:37.026907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93e475-5cab-4115-bccd-1d73267c9944 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:37.028069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93e475-5cab-4115-bccd-1d73267c9944 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:37.029511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93e475-5cab-4115-bccd-1d73267c9944 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:37.144491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 478/960] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:37.186132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65664-7a30-4946-aa6b-0f3526cbe697 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:37.186708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65664-7a30-4946-aa6b-0f3526cbe697 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:37.186708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65664-7a30-4946-aa6b-0f3526cbe697 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:37.188243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 483/961] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:37.193765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:37.194052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:37.194329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:37.214782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:37.216352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:37.305131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 479/962] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:37.329423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a65664-7a30-4946-aa6b-0f3526cbe697 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:37.452644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a65664-7a30-4946-aa6b-0f3526cbe697 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:37.467186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:37.488875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:37.489480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:37.489758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:37.490027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:37.490691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:37.492109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:37.494676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING keystone.server.flask.common [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 22:53:37.548712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.sql.core [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 22:53:37.548712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 22:53:37.548712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [parameters: {'user_id': '52a4ab4656d74d82b6b71ec9afcad68b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 22:53:37.548712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81638) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 10 22:53:37.566267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:53:37.567302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application yield resource Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise self.value Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application session.commit() Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 22:53:37.568742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.flush() Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application compat.raise_( Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 22:53:37.569752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:53:37.570638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:53:37.571560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [parameters: {'user_id': '52a4ab4656d74d82b6b71ec9afcad68b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:37.572812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 22:53:37.573681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:37.575431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 484/963] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 10 22:53:37 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:37.582731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:37.634085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:37.684329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20e83e-982e-42c6-b9ee-66e7939b4d52 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:37.724357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:37.724901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:37.725178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:37.725447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:37.733045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:37.734540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f10a84ff-05a5-4ffe-bc44-276196fadebd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:37.746531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:37.749032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:37.751462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 480/964] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:37.755488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:37.859890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57457bf-3fb8-4142-9186-25fadae332cb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:37.896167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:37.896786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:37.897066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:37.897338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:37.898029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:37.898330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:37.899998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589573f7-d85c-4189-a333-c279ffbaf9d0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:37.938920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 485/965] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 10 22:53:37 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:38.101256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4fad5-b5f4-4c40-9e89-97de988d0709 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.101743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4fad5-b5f4-4c40-9e89-97de988d0709 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.101743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4fad5-b5f4-4c40-9e89-97de988d0709 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.103450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 481/966] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:38.109838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.110117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.110423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.139644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:38.141309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:38.172194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:38.217018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 486/967] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:38.240466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4fad5-b5f4-4c40-9e89-97de988d0709 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.345971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4fad5-b5f4-4c40-9e89-97de988d0709 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.361225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:38.382939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.383558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.383833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.384099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.396246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:38.401189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:38.402702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:38.403624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:38.410979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 482/968] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 10 22:53:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:38.412096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.519543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4c479-e3ba-445a-9cd2-f5ec4e401e0e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.553197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.553790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.554062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.554326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.555009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:38.555305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:38.556654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:38.564719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 487/969] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:53:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:38.570098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.671303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee6567-4b8f-4fb5-87ad-67c8e58e663d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.704784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.705394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.705664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.705924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.717103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:38.718525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:38.732874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:38.733918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:38.735259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:38.735259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:38.735259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:38.735713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 483/970] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:38.739571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.766969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:38.842183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9204a-b259-4d3b-af7e-8d8c788d3109 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.875759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.876353 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.876626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.876891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.877517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:38.877811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:38.879182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86483031-0baa-4977-8a65-d77c808f80fc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:38.896000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 488/971] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:38.897628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.925090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.000273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eeedaf-3cd7-41dd-9cb0-6ba3dc1ad89c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.033764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.034385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.034672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.034957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.042521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31243f-c42a-4f3f-88f3-9556da4acfad None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.042691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.043461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31243f-c42a-4f3f-88f3-9556da4acfad None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.043701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31243f-c42a-4f3f-88f3-9556da4acfad None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.044741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.046705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 489/972] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:39.049842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.050116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.050382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:39.059386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:39.060382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:39.061380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 484/973] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:53:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:39.066539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.070544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:39.071749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:39.144002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 490/974] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:39.164840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.166410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.182465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06a056-0a86-44a5-abc8-2312b48441d4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.183474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.217085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.222415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.222415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.222415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.222415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:39.222415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.222415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.239350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 485/975] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:39.245073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.271609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58eea1-cac3-4b7f-8a67-733d44cab295 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.307421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de497b1e-28d3-4d60-9252-91ac61759506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.308131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497b1e-28d3-4d60-9252-91ac61759506 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.308409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497b1e-28d3-4d60-9252-91ac61759506 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.308686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497b1e-28d3-4d60-9252-91ac61759506 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.309323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de497b1e-28d3-4d60-9252-91ac61759506 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.310698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de497b1e-28d3-4d60-9252-91ac61759506 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.348956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c269-ed90-4f1e-a636-88cd9dad8704 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.381477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.382069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.382341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.382606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.394421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.447012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.448398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.469135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 491/976] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:39 2022] POST /identity/v3/services => generated 242 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:39.470381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577935e6-dd19-4183-87d8-5c3ad84d92ee None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.470680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577935e6-dd19-4183-87d8-5c3ad84d92ee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.470966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577935e6-dd19-4183-87d8-5c3ad84d92ee None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.474820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 492/977] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:39.484477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.484793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.485160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.507609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:39.510649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:39.569984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 486/978] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:53:39.577280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.594098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.600912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 493/979] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:39.625296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.627323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.687583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959449b-ccc0-4830-a59a-ea380d130c25 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.724441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.724968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.725243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.725510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.731438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ad5d9-ec9c-440a-affa-9fc431859622 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.736769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.738219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:39.754555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:39.755579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:39.756580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:39.756580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:39.756580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:39.756580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 487/980] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:39.761396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.772693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83183a-3658-4529-baea-415fb394859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.773321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83183a-3658-4529-baea-415fb394859f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.773594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83183a-3658-4529-baea-415fb394859f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.774094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83183a-3658-4529-baea-415fb394859f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.774548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83183a-3658-4529-baea-415fb394859f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.776045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83183a-3658-4529-baea-415fb394859f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1a83183a-3658-4529-baea-415fb394859f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:39.785597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:39.786594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:39.788230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 494/981] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:39.792640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83183a-3658-4529-baea-415fb394859f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.858288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.864823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:39.872496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170650fe-dd7c-4892-ac31-e0a52e6c96e2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.900397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83183a-3658-4529-baea-415fb394859f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:39.907588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.908199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.908480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.908747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.909388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:39.909687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.911103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.923449 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 488/982] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:39.928939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:39.943565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:39.944184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:39.944462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:39.944731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:39.945360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:39.945661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:39.947053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.956388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 495/983] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:39.957725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:40.038070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930a223-e7a5-492f-8da5-5d6f5b1411f1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.064253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67789d-37ca-4a17-ba19-5f85b9060bf2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.074924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.075545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.075822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.076088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.083233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.084679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.089912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:40.096736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:40.097887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:40.099263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 489/984] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:40.103289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.103567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:40.103943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.104171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.104543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.105778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:40.105984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.107384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.112484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 496/985] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:40.118231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:40.120300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:40.213222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2d42-17f2-4c7f-ac5c-70d6442c4a1f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.237033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d830-9e67-4f7d-b0ab-280d8e20017f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.251278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.252247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.252674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.252940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.253619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:40.253913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.255343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.267739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 490/986] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:40.273140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.273622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:40.273724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.274000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.274260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.274983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] RBAC: Adding query filter params (interface=admin, service_id=f5ffab5ee74c4a8992c35771bb2fe7ee, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:40.275286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.276735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.282725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 497/987] 10.222.0.78 () {56 vars in 1249 bytes} [Wed Aug 10 22:53:40 2022] GET /identity/v3/endpoints?service_id=f5ffab5ee74c4a8992c35771bb2fe7ee&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:40.288031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:40.346023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:40.358280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:40.385811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27c1a7-4a6c-4bb2-9554-d1784e3acfe8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.394135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e002e8-bd93-4098-aef5-daae27b3038b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.427479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.428567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.428938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.429306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.430152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] RBAC: Adding query filter params (role.id=2ca8c476a62e417c86da67ab92dbf61c, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=bbf97a6003294b5ba411fa9ad2fda18d) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:40.430558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.432125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e7dd2-79df-4ef4-ab41-3e54cb04644a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.437036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8a2cd-590d-42ad-b52d-affd5ca30bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.437645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8a2cd-590d-42ad-b52d-affd5ca30bad None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.437924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8a2cd-590d-42ad-b52d-affd5ca30bad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.438196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8a2cd-590d-42ad-b52d-affd5ca30bad None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.438802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 491/988] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:40 2022] GET /identity/v3/role_assignments?user.id=bbf97a6003294b5ba411fa9ad2fda18d&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=2ca8c476a62e417c86da67ab92dbf61c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:40.438926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8a2cd-590d-42ad-b52d-affd5ca30bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.440166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8a2cd-590d-42ad-b52d-affd5ca30bad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.446948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 498/989] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:40 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:53:41.528830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95173e-c800-4f12-8554-7bc7b13f3ca8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:41.529397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95173e-c800-4f12-8554-7bc7b13f3ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:41.529502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95173e-c800-4f12-8554-7bc7b13f3ca8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:41.530968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 492/990] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:41.536329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:41.536786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:41.536871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:41.555270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:41.556482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:41.631255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 499/991] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:41.654341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95173e-c800-4f12-8554-7bc7b13f3ca8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:41.754447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95173e-c800-4f12-8554-7bc7b13f3ca8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:41.787575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:41.788181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:41.788454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:41.788721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:41.789288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:41.790636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:41.799110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:41.800198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:41.801627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:41.808719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 493/992] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:41.808868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:41.867301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:41.918180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b4578-ebf6-498e-8776-f056255d11f0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:41.954568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:41.955481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:41.955939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:41.956363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:41.957140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:41.957650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:41.959226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:41.964546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 500/993] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:41.969795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:41.973362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:42.074155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aff659-f866-4384-9978-3b831c5685d3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.109796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.110398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.110681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.110968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.111590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:42.111889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.113252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.123238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 494/994] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:42.124680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.252464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9320d-64df-458b-adee-ac30d5584267 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.267224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:42.287580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.288182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.288458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.288727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.289613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] RBAC: Adding query filter params (interface=public, service_id=4cf05cf84a2b4c8eaaee2f699743f0f9, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:42.289929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.291298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee3b32-b27d-4373-aec7-2a373675f231 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.303008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 501/995] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/endpoints?service_id=4cf05cf84a2b4c8eaaee2f699743f0f9&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:42.303182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.370626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:42.405678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19fd71-db31-442e-a998-66335a3bd8dd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.448914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e80a7-84a2-4d9c-bf1f-fb79d4407253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.449805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e80a7-84a2-4d9c-bf1f-fb79d4407253 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.450242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e80a7-84a2-4d9c-bf1f-fb79d4407253 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.450687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e80a7-84a2-4d9c-bf1f-fb79d4407253 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.451430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e80a7-84a2-4d9c-bf1f-fb79d4407253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.452936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e80a7-84a2-4d9c-bf1f-fb79d4407253 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.459927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 495/996] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:53:42.485876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749843-786d-4960-95ed-8948d5728758 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.486192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749843-786d-4960-95ed-8948d5728758 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.486423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7749843-786d-4960-95ed-8948d5728758 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.488054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 502/997] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:42.493736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.494085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.494272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.514242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:42.515561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:42.530699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2c21-caf0-4ab6-b706-1891c27e0221 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.531031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2c21-caf0-4ab6-b706-1891c27e0221 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.531240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2c21-caf0-4ab6-b706-1891c27e0221 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.532852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 503/998] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:42.538423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29292a7-b328-4551-8346-06f89527a470 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.539094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29292a7-b328-4551-8346-06f89527a470 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.539094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29292a7-b328-4551-8346-06f89527a470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.557597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e29292a7-b328-4551-8346-06f89527a470 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:42.558878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e29292a7-b328-4551-8346-06f89527a470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:42.606731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 496/999] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:42.627548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.635398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 504/1000] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:42.659763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29292a7-b328-4551-8346-06f89527a470 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.692125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:42.736592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dddce8e-01f6-42d7-802b-de4c76a7082d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.770614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29292a7-b328-4551-8346-06f89527a470 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.775199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.775891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.776176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.776450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.777552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.779379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.781071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:42.790711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:42.795412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:42.796516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 497/1001] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:42.798236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.818069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.818997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.819451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.819887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] PATH_INFO: `/v3/services/image` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.821153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.822949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:42.831733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:42.833018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:42.834100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:42.836283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 505/1002] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:42.842631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.908055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4afbd-4654-40bf-980e-ae8c3feebe76 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.933279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:42.944969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.945292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd9837-2212-4a23-9617-14890029b1db None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:42.945641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.945918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.946189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.947001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:42.947301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.949428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.954665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 498/1003] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:42.960320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:42.983780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:42.984386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:42.984665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:42.984941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:42.985568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:42.985867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:42.987275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:42.992194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 506/1004] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:42.997816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:43.035589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:43.065655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f3b6-3f82-41e8-b621-95f9842d1492 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:43.099728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:43.103376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:43.103376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:43.103376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:43.103376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:43.103376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:43.103376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:43.103805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66959e8-9978-4937-9bca-cab934308dac None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:43.108316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 499/1005] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:43.113709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:43.138737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:43.139350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:43.139992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:43.140470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:43.141748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:43.142214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:43.143755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c30aa4-fd80-4e79-a32e-ce4e49d1b9f5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:43.148992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 507/1006] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:43.174220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:43.227135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd9753-f326-4d5b-b52e-8e48fd992c7d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:43.264395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d540b9-17ae-4859-9f7a-25da6b5797f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:43.265026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d540b9-17ae-4859-9f7a-25da6b5797f7 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:43.265305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d540b9-17ae-4859-9f7a-25da6b5797f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:43.265572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d540b9-17ae-4859-9f7a-25da6b5797f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:43.266235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d540b9-17ae-4859-9f7a-25da6b5797f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:43.267660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d540b9-17ae-4859-9f7a-25da6b5797f7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:43.349397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 500/1007] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:44.501501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e123c7-e7ba-4d74-9365-265e2632d1df None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:44.502049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e123c7-e7ba-4d74-9365-265e2632d1df None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:44.502049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e123c7-e7ba-4d74-9365-265e2632d1df None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:44.503546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 508/1008] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:44.508526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:44.508984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:44.509183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:44.529208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:44.530393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:44.601236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 501/1009] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:44.623100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e123c7-e7ba-4d74-9365-265e2632d1df None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:44.723619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:44.728033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e123c7-e7ba-4d74-9365-265e2632d1df None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:44.760856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:44.761993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:44.762265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:44.762530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:44.763132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:44.764516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:44.773089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:44.774186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:53:44.775509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:44.778971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 509/1010] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:44.780607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:44.823275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:44.883711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee77e3-e51b-467c-9858-dd09e92713b0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:44.918638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:44.919304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:44.919577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:44.919841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:44.920455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:44.920765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:44.922179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:44.931377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 502/1011] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:44.932633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:45.036374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454bec5-f8d7-43ec-8868-7cf4fad1e267 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:45.070480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:45.071111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.071436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.071705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.072326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:45.072619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:45.074512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914f561-f971-45be-9379-47a1eebe47d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:45.084283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 510/1012] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 10 22:53:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:45.085214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:45.186999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5936-e1bc-4e2d-a99b-4b4362cf5ec1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:45.188547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:45.191122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be9b1-194a-4dcc-bdf0-b8586a257559 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.191404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be9b1-194a-4dcc-bdf0-b8586a257559 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.191679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be9b1-194a-4dcc-bdf0-b8586a257559 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.193150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 511/1013] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:45.197875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.201972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.201972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.216646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:45.217817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:45.222997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa54bd-70bf-4d90-bfcc-3e5a0f618827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:45.223597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa54bd-70bf-4d90-bfcc-3e5a0f618827 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.223872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa54bd-70bf-4d90-bfcc-3e5a0f618827 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.224140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa54bd-70bf-4d90-bfcc-3e5a0f618827 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.224881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa54bd-70bf-4d90-bfcc-3e5a0f618827 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:45.226175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa54bd-70bf-4d90-bfcc-3e5a0f618827 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:45.242080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:45.285888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 512/1014] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:45.309023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:45.331543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 503/1015] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:45.401314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562bcd9-ed47-4f64-87f9-654a7c13842e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.401619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562bcd9-ed47-4f64-87f9-654a7c13842e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.401892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562bcd9-ed47-4f64-87f9-654a7c13842e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.403661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 504/1016] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:45.409189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.409502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.409749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.423307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878eef01-1eb7-4a4f-ae6a-029311ce5d6e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:45.434582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:45.435831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:45.470029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:45.470645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.471076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.471352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.472006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:45.473398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:45.509202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 505/1017] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:45 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:45.532554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:45.621780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 513/1018] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:45 2022] POST /identity/v3/services => generated 242 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:45.624257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:45.639414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ccb6b-6263-4c45-a275-43bcad9c424d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:45.684976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf0865-1b2e-47b6-9437-f196005de7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:45.685705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf0865-1b2e-47b6-9437-f196005de7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.686052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf0865-1b2e-47b6-9437-f196005de7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.686390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf0865-1b2e-47b6-9437-f196005de7c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.687657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf0865-1b2e-47b6-9437-f196005de7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:45.689276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf0865-1b2e-47b6-9437-f196005de7c7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:45.691303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 506/1019] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:45.697042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:45.717394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:45.813234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ecca9-1c40-46bf-ac87-1c9745968954 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:45.848955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cceb703-de6f-410e-9a35-d10d99a4552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:45.849653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceb703-de6f-410e-9a35-d10d99a4552e None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:45.850000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceb703-de6f-410e-9a35-d10d99a4552e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:45.850338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceb703-de6f-410e-9a35-d10d99a4552e None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:45.851080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cceb703-de6f-410e-9a35-d10d99a4552e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:45.852559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cceb703-de6f-410e-9a35-d10d99a4552e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:45.961914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 514/1020] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:53:45 2022] POST /identity/v3/projects => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:47.492451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54f072-d256-4dd1-82fd-ff4ed0c5cce1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.492967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54f072-d256-4dd1-82fd-ff4ed0c5cce1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.492967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54f072-d256-4dd1-82fd-ff4ed0c5cce1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.494460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 507/1021] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:47.500298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fcc47-c6b1-4c0c-ab2c-3663727e11ca None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.500720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fcc47-c6b1-4c0c-ab2c-3663727e11ca None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.500896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fcc47-c6b1-4c0c-ab2c-3663727e11ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.519318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1d6fcc47-c6b1-4c0c-ab2c-3663727e11ca None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:47.520528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d6fcc47-c6b1-4c0c-ab2c-3663727e11ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:47.586498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:47.594571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 515/1022] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:47.617877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54f072-d256-4dd1-82fd-ff4ed0c5cce1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:47.694192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9fe0-f16c-4909-9b66-dc774e165563 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.694382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9fe0-f16c-4909-9b66-dc774e165563 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.694639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9fe0-f16c-4909-9b66-dc774e165563 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.696162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 516/1023] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:47.701064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.701346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.701622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.720307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:47.721752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:47.725253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54f072-d256-4dd1-82fd-ff4ed0c5cce1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:47.760815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:47.761465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.761743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.762013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.772267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:47.777448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:47.778447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:47.780177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 508/1024] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 10 22:53:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:47.784375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:47.800610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:47.801704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 517/1025] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:47.826601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:47.890110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5da28-e43a-4ab1-9075-ce0c117a415d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:47.927204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:47.927807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.928079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.928340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.937238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:47.937238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:47.937238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:47.937466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788d98-76bf-48ca-9425-ac7bf0d7e3ef None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:47.939146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 509/1026] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:53:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:47.945004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:47.971264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:47.980551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:47.981188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:47.981468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:47.981738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] PATH_INFO: `/v3/services/image` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:47.985317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:47.985317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:47.992677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:47.993720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:47.994624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 518/1027] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Aug 10 22:53:47 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:47.999984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.027614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.052333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc77a3-899d-479e-a3e1-29eae02def43 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.085782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.086392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.086720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.086989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.097360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.098798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.100743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6d8f3-8f1e-4eef-8c91-36779994ae10 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-df1a410c-eae0-4777-bdd7-0a7b31b476d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:48.112933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:48.113971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:48.115521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:48.115521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:48.115521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:48.115521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 510/1028] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:48.115521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a8f9-af8b-4832-ace1-0d6278f691c7 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.115521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a8f9-af8b-4832-ace1-0d6278f691c7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.115898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a8f9-af8b-4832-ace1-0d6278f691c7 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.117184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 511/1029] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:48.119696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a8f9-af8b-4832-ace1-0d6278f691c7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.135431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.136053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.136978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.137252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.137872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:48.138168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.139554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e0ba-e196-4e5b-bfca-3ed02fa8413f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.144243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 519/1030] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:48.145355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.145640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.145911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.163048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:48.164230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:48.212919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.228395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a8f9-af8b-4832-ace1-0d6278f691c7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.232067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 520/1031] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:48.232898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.256020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.264342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d449049-324c-4542-8075-675931535153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.265011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d449049-324c-4542-8075-675931535153 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.265348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d449049-324c-4542-8075-675931535153 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.265549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d449049-324c-4542-8075-675931535153 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.266325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d449049-324c-4542-8075-675931535153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:48.266616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d449049-324c-4542-8075-675931535153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.267996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d449049-324c-4542-8075-675931535153 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.279313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 512/1032] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:48.282928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d449049-324c-4542-8075-675931535153 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.335601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc5305-dff3-4edc-81e1-d1b141a3e8e6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.376902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.377510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.377787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.378059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.378961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] RBAC: Adding query filter params (type=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:48.379271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.380639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.385734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 521/1033] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:48.387371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.387501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d449049-324c-4542-8075-675931535153 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.427277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.427967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.428236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.428553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.430707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.432130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.434003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 513/1034] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:48.434840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.440651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.489002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.492032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44447405-4ac8-496e-99da-9a913e7f1258 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.528453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.529076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.529327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.529594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.533987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20720d-dc6d-4522-8991-7ed74ea4a56f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.536490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.537912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:48.549047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:48.550062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 522/1035] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:48.551124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.572318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.572836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.573110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.573374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.574612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] RBAC: Adding query filter params (interface=public, service_id=e586c32acf9b40c3b14373a112c29343, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:48.574955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.576295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.581355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 514/1036] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/endpoints?service_id=e586c32acf9b40c3b14373a112c29343&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:48.582121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.651787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276e8cd-a158-497b-b7b5-69b1e66ea602 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.660647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.679943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a768e7-0ccc-47de-aca7-5ab737c3c960 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.686156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.686787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.687074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.687344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.687983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.689396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.697631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:48.715120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.715707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.715978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.716243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.716949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:48.717247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.718674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.735590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 515/1037] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:48.736246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.809537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 523/1038] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:53:48 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:53:48.810373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:48.838028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa63fe-f2f4-49b4-a449-53f5d2bcdc0d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.885860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.887222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.887684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:48.888029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:48.888650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:48.890057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:48.906686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 516/1039] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:53:48.956401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213da67-36e5-4170-b5de-7624900e6aee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:48.999596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:48.999596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:48.999876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:49.000133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:49.038646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:49.075837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:49.077535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:49.182370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 524/1040] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:53:49.184636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:49.226787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:49.285447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0936dbc-b785-4ee7-acb6-9eda4686f880 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:49.319637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:49.320348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:49.320693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:49.321032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:49.332213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:49.333667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:49.349801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:49.351059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:49.351975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:49.351975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:49.351975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:49.351975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 517/1041] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:49.355098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:49.434782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:49.465505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd823ec-5765-4bcc-bbe9-6d6d282650e0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:49.500700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:49.501302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:49.501581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:49.501850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:49.502486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:49.502820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:49.504283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:49.520677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 525/1042] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:49.521811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:49.621085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:49.630638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd69b1-014a-4f79-b1e2-1cacddc41ad3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:49.664894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:49.665509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:49.665783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:49.666052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:49.673219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:49.674638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:49.686024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:49.687181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:49.688430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:49.689318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 518/1043] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:49.692960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:49.796027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cc4ee-c005-4db0-aeb6-e67178f4615b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:49.818964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:49.829717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:49.830317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:49.830594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:49.830893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:49.831574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:49.831874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:49.833347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f77250-9f0d-4f2a-ab87-3d20ec27ac54 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:49.850012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 526/1044] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:49.851246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:49.964560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ad7ec-96d4-4b3f-b66d-9cadafcfae83 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:50.002788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:50.003409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:50.003691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:50.003958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:50.004722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] RBAC: Adding query filter params (role.id=2ca8c476a62e417c86da67ab92dbf61c, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=07cef180b05347ebbcc6f3a6e97dac3a) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:50.005028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:50.006476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2bcbc-20f0-47d9-90a9-d0d12c99626c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:50.012531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 519/1045] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:49 2022] GET /identity/v3/role_assignments?user.id=07cef180b05347ebbcc6f3a6e97dac3a&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=2ca8c476a62e417c86da67ab92dbf61c => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:50.903568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20c411-8914-49d5-8369-4963c0285b5d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:50.904090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20c411-8914-49d5-8369-4963c0285b5d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:50.904170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20c411-8914-49d5-8369-4963c0285b5d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:50.905592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 527/1046] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:50.910506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:50.910944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:50.911103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:50.928888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:50.930069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:50.933876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16783953-0a9a-4c2a-be29-479ae75de932 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:50.934557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16783953-0a9a-4c2a-be29-479ae75de932 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:50.934557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16783953-0a9a-4c2a-be29-479ae75de932 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:50.935922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 528/1047] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:50.940511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:50.940786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:50.941104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:50.941988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:50.967656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:50.968849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:51.010007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 520/1048] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:51.033231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.042491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 529/1049] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:51.065121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.151162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8216-49af-4933-a5e2-ac112b4544bd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.172080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9a66e-0f9d-4d3a-8469-31662b91c7eb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.186080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:51.194274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.199065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.199065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.199065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.206526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:51.206708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.207204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.207480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.207744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] PATH_INFO: `/v3/services/image` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.208308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.209892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:51.211243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:51.212171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:51.213035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:51.213699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 521/1050] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:51.219278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:51.224225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:51.225244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:51.226104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 530/1051] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:51.229737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:51.231175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.335322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56c02d-d211-4efb-8e64-0238965c3610 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.351073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc32e98-187a-44cd-b43a-06701a5f3809 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.371083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.371986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.372424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.372871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.374069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:51.374517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.376143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.384382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.384988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.385212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 522/1052] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:51.385309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.385510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.386647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:51.386969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.388309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.391313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.394707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 531/1053] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:51.400120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.449371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:51.474357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:51.509214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f5a8-6f80-4c38-92d7-14e9eef27ecb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.514420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac9a5f-acbe-4e21-af23-aff7baf758d0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.545777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.546398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.546704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.546982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] PATH_INFO: `/v3/users/9ca7b16d991446808d84a12480134e77` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.549114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.549719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.549990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.550256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.550903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] RBAC: Adding query filter params (type=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:51.551202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.552557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.557593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 532/1054] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:51.562276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] RBAC: Authorizing `identity:get_user(user_id=9ca7b16d991446808d84a12480134e77)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.563817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.564224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.565612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 523/1055] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/users/9ca7b16d991446808d84a12480134e77 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:51.571391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:51.685759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b525-46b4-4490-badd-96ec112b0cdd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.688226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030d2a5-7183-401c-96ff-425f13f9d4cf None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:51.693254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:51.716104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:51.722212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.722850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.723126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.723389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.724200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.725662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.725777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f00106-4bf6-4117-b964-501014604510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:51.726281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f00106-4bf6-4117-b964-501014604510 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:51.726562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f00106-4bf6-4117-b964-501014604510 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:51.726862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f00106-4bf6-4117-b964-501014604510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:51.795218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f00106-4bf6-4117-b964-501014604510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:51.796726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f00106-4bf6-4117-b964-501014604510 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:51.857114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 533/1056] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:51.905401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 524/1057] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:53:51.911447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.034002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4010f-fc58-4d3d-802d-2067d575b5a3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.034503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4010f-fc58-4d3d-802d-2067d575b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.034503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4010f-fc58-4d3d-802d-2067d575b5a3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.035831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 525/1058] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:52.036397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd903386-c284-40d6-8fa9-352d4f8fd080 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.040916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.041193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.041471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.062770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:52.064586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:52.077275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.078209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.078783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.079064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.080202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=5c837b44ffb7450491045945971ed005)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:52.082726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:52.084335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 534/1059] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:53:51 2022] GET /identity/v3/projects/5c837b44ffb7450491045945971ed005 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:52.087677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:52.090500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.120062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:52.139702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 526/1060] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:52.161701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.196146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f658-3dcf-4e50-8956-679a2f942ba1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.232282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.234024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.234305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.234569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.236091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=5c837b44ffb7450491045945971ed005, user.id=9ca7b16d991446808d84a12480134e77) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:52.236401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:52.237822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:52.254698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 535/1061] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:52 2022] GET /identity/v3/role_assignments?user.id=9ca7b16d991446808d84a12480134e77&scope.project.id=5c837b44ffb7450491045945971ed005&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:52.268252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4a84c-c111-43f2-bf5f-2c8a3944f7cc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.308447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.309005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.309279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.309546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.321321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:52.326697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:52.327699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:52.333894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 527/1062] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 10 22:53:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:52.336403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.440831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:52.450840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea691c8c-e98c-4734-bbc5-b45cf198e808 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.484015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.484612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.484890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.485142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.485788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:52.486082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:52.487450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:52.496092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 536/1063] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:53:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:52.502117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.521631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:52.603812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82987ec-ffd7-426b-96da-b0c061fab60b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.639114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.639725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.639994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.640257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.650120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:52.652093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:52.666336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:52.668082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:52.669035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:52.669035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:52.669035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:52.670730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 528/1064] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:52.673993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.775321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf29a-b414-487d-9830-f9ff9bdfdd22 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.810205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.810831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.811110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.811379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.812007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:52.812306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:52.813745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:52.830268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 537/1065] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:52.832044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:52.904203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:52.937584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621f35a-70b9-4a6b-8712-ef1060066764 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:52.972688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:52.973313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:52.973593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:52.973864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:52.982615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:52.984084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:52.995258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:52.996839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:52.997795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:53.001907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 529/1066] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:53:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:53.003390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:53.018600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:53.110346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab55ae7-d744-4807-85ae-4420e08384c3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:53.144704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:53.145295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.145566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.145826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.146671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:53.146869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:53.148250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:53.169509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 538/1067] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:53:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:53.171300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:53.273617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc85a2-fa19-4168-9079-b893457380f2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:53.299007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:53.308239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:53.308861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.310283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.310283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.320024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:53.321459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:53.336370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:53.337355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:53.338292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:53.338292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:53.338292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:53.339036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 530/1068] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:53.343511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:53.416604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:53.453848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3e5e8-98ce-439b-955c-5dbfb634f284 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:53.489047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:53.489644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.490016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.490192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.490834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:53.491132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:53.492485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:53.509070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 539/1069] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:53.510365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:53.609023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1949e-3468-4ecd-bb9c-7ec150145692 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:53.643418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:53.644131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.644475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.644812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.651601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:53.653100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:53.664128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:53.665225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:53.666452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:53.672486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 531/1070] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:53.673508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:53.773542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bebb9-d982-4425-b0aa-120f8f52c404 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:53.803940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:53.810588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:53.811294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.811625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.812006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.813519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:53.813519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:53.814649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629eb41-18dd-4f90-9eb5-94b6b4af6c91 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:53.831533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 540/1071] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:53.832935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:53.876156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:53.936693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b438cc-0520-4a27-97c4-ffaa4f52c94c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:53.944389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb06ef-a57d-48fc-bb08-44d523cbafac None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.944669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb06ef-a57d-48fc-bb08-44d523cbafac None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.944945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb06ef-a57d-48fc-bb08-44d523cbafac None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.946700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 541/1072] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:53:53.951283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.951558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.951826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.971342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:53.972530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:53.976761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fab19f-e11c-4e63-9823-54f02a21a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:53.977286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab19f-e11c-4e63-9823-54f02a21a08f None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:53.977563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab19f-e11c-4e63-9823-54f02a21a08f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:53.977837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab19f-e11c-4e63-9823-54f02a21a08f None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/07cef180b05347ebbcc6f3a6e97dac3a/roles/2ca8c476a62e417c86da67ab92dbf61c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:53.980623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fab19f-e11c-4e63-9823-54f02a21a08f None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=07cef180b05347ebbcc6f3a6e97dac3a, role_id=2ca8c476a62e417c86da67ab92dbf61c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:53.982361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fab19f-e11c-4e63-9823-54f02a21a08f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:54.040438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 542/1073] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:54.063336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:54.148033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 532/1074] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:53 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/07cef180b05347ebbcc6f3a6e97dac3a/roles/2ca8c476a62e417c86da67ab92dbf61c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:53:54.174299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833231a-ec24-490f-b229-89740cc4a3c3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:54.197596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:54.213581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:54.214305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.214588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.214959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.294372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:54.295777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:54.301965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e069ee7-1dcc-485b-8f99-ab7f6f66ef10 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.301965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e069ee7-1dcc-485b-8f99-ab7f6f66ef10 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.302542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e069ee7-1dcc-485b-8f99-ab7f6f66ef10 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.304714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 533/1075] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:54.311700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.312241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.312578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.330893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:54.332078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:54.406499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 534/1076] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:54.417628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 543/1077] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:53:54.419444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:54.431078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:54.452942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:54.454792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:54.528875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a6847-93dd-4bb7-beb8-4ced58330ade None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:54.535070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58675226-4897-45a7-9d3c-518c3ec515a5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:54.564531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:54.565082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.565350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.565614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.569954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:54.570578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.570878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.571147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.576174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:54.577620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:54.582413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:54.588421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:54.589940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:54.591017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 544/1078] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:54.594253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:54.599877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:54.600988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:54.600988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:54.600988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:54.601178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:54.601254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 535/1079] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:54.605129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:54.689350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:54.697514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:54.710299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973e392-022a-44a9-af2f-6c4d5a8ea3ef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:54.716649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1cbad-5f4f-4cab-9036-b4fc2c382f65 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:54.745886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:54.746490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.746809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.753807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.753807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:54.753807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:54.753807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:54.754150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:54.754150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.754150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.754150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.754774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:54.755070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:54.756416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:54.759365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 545/1080] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:54.764763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:54.770686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 536/1081] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:54.887242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725309d-b67c-411d-91ef-2ac9a36e3275 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:54.926159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:54.931547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:54.931547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:54.931547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] PATH_INFO: `/v3/users/9ca7b16d991446808d84a12480134e77` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:54.931547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] RBAC: Authorizing `identity:get_user(user_id=9ca7b16d991446808d84a12480134e77)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:54.931547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:54.932076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 546/1082] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/users/9ca7b16d991446808d84a12480134e77 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:54.937504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:55.049364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49124a00-3789-4254-a065-1171bf6a608f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:55.083638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba1f07-5a85-462c-869f-3fb33be31942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:55.084254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba1f07-5a85-462c-869f-3fb33be31942 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:55.084525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba1f07-5a85-462c-869f-3fb33be31942 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:55.084789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba1f07-5a85-462c-869f-3fb33be31942 None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:55.085898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba1f07-5a85-462c-869f-3fb33be31942 None admin] RBAC: Authorizing `identity:get_project(project_id=5c837b44ffb7450491045945971ed005)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:55.087333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba1f07-5a85-462c-869f-3fb33be31942 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:55.088961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 537/1083] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:53:54 2022] GET /identity/v3/projects/5c837b44ffb7450491045945971ed005 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:55.095018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:55.107808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:55.196344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d6b6-d6ee-4967-bc5b-126daebfad03 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:55.229165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda93799-8566-409f-ae6a-91e3f53fc25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:55.230338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda93799-8566-409f-ae6a-91e3f53fc25f None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:55.230613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda93799-8566-409f-ae6a-91e3f53fc25f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:55.230916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda93799-8566-409f-ae6a-91e3f53fc25f None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005/users/9ca7b16d991446808d84a12480134e77/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:55.232817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda93799-8566-409f-ae6a-91e3f53fc25f None admin] RBAC: Authorizing `identity:create_grant(project_id=5c837b44ffb7450491045945971ed005, user_id=9ca7b16d991446808d84a12480134e77, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:55.234236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda93799-8566-409f-ae6a-91e3f53fc25f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:55.348203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 547/1084] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:53:55 2022] PUT /identity/v3/projects/5c837b44ffb7450491045945971ed005/users/9ca7b16d991446808d84a12480134e77/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:56.191533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438d58-a6e2-44df-8b4e-c3fb8ff5aad6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:56.192096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438d58-a6e2-44df-8b4e-c3fb8ff5aad6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:56.192096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438d58-a6e2-44df-8b4e-c3fb8ff5aad6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:56.193545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 538/1085] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:56.198770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:56.199196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:56.199328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:56.218038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:56.219233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:56.279448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:56.283889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 548/1086] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:53:56.306997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68438d58-a6e2-44df-8b4e-c3fb8ff5aad6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:56.329642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:56.419201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68438d58-a6e2-44df-8b4e-c3fb8ff5aad6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:56.457322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:56.457863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:56.458142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:56.458411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:56.468396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:56.473019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:56.474015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:56.474940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 539/1087] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 10 22:53:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:56.480645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:56.576043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d50e7-c7fe-4533-b01b-7395c8a98bfa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:56.608714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:56.609301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:56.609571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:56.609834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:56.610477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:56.610798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:56.612147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:56.619696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 549/1088] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:53:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:56.625707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:56.694883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:56.725228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518bc83-80c7-46fe-b547-45faee318d73 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:56.758938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:56.759551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:56.759828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:56.760106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:56.770454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:56.771928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:56.786477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:56.787502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:56.788435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:56.788435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:56.788435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:56.788435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 540/1089] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:56.793365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:56.854470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:56.893085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da684d89-f1b3-459a-b702-be9760a45426 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:56.927657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:56.928252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:56.928528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:56.928800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:56.929433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:56.929729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:56.931171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:56.943499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 550/1090] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:56.949676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.048200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846141c5-26d0-4ec1-b74b-87c45c10af6f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.069998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.081642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.084651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.084651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.084651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.090059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.091511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:53:57.102139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:57.103131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:57.104098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 541/1091] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 10 22:53:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:57.109097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.208052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8bf67-636d-4823-b59b-e4c55389094d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.230521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.241795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.242399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.242748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.253577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.253577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:57.253577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.253577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc862947-e867-4c96-b6f9-72ef944212af None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.268155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 551/1092] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:57.269334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.321055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf0ae9-8de9-449b-8316-268d17e45ad6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.321240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf0ae9-8de9-449b-8316-268d17e45ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.321516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf0ae9-8de9-449b-8316-268d17e45ad6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.325298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 552/1093] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:53:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:53:57.330480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.330792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.331067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.348579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:57.349770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:57.375553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b89c-84db-4c8e-9ed5-9604b840fc04 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.407721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.408312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.408586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.408851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.417917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 553/1094] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:53:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:53:57.441659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.463346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.480948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.482393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.538588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274826-e022-4755-9b5a-62b23e695b74 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.549651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.573761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.574323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.574598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.574892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.579155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 542/1095] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:53:57.584960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.585088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:53:57.589908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:57.591055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:53:57.591923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 554/1096] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:57.596826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.665505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.690031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10104dec-38c2-4a91-8203-09a7f01a9f3c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.703589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce8c72-67d5-46c7-82a0-593b5ae938fe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.725416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.726035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.726316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.726583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.736477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.736961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.737558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.737833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.737929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.738101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.738784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:57.739083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.740498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.749545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 555/1097] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:53:57.753119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:57.754260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:53:57.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:53:57.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:53:57.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:57.755204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 543/1098] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:53:57.755709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.759585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.787108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.854924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9025fe-5e8d-4deb-aba8-d1888cd724b5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.858901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514f897-7777-4f4c-a2bd-bd36b4982c2b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:57.888412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.889034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.889308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.889578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] PATH_INFO: `/v3/users/9ca7b16d991446808d84a12480134e77` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.890775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] RBAC: Authorizing `identity:get_user(user_id=9ca7b16d991446808d84a12480134e77)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.892193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.892482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:57.893065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:57.893339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:57.893605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:57.898690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 556/1099] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/users/9ca7b16d991446808d84a12480134e77 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:57.898831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:57.898831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:57.898831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:57.898831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.899678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.909136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 544/1100] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:57.914592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:57.994974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:57.998104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb78f7-fb58-4f92-bbf8-60a454921402 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:58.015941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fdcc6-bfbe-4104-9b26-9217fdf4bef1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:58.033465 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:58.034074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:58.034345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:58.034613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:58.058342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:58.058996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:58.059269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:58.059534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:58.066361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:58.067830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:53:58.079107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:53:58.080220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:53:58.081528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 545/1101] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:53:58.085910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:58.108099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:58.109602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:58.140542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:58.185036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f6f5-43ba-4849-85eb-f8ce7fceda1f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:58.204552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:58.208099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 557/1102] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:53:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:53:58.214070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:58.219930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:58.220589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:58.226507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:58.226507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:58.226507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:58.226507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:58.226507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:58.235410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 546/1103] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:58.241568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:58.316593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab96c-4e0f-4f42-83d0-37802e362155 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:58.347140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9498-ae94-475e-a41d-c480cf706687 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:58.356219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:58.357322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:58.357762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:58.358289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:58.359885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=5c837b44ffb7450491045945971ed005)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:58.362235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:58.365147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 558/1104] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:53:58 2022] GET /identity/v3/projects/5c837b44ffb7450491045945971ed005 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:58.369138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:58.374811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:58.391881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:58.392486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:58.392765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:58.393034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:58.393745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] RBAC: Adding query filter params (role.id=2ca8c476a62e417c86da67ab92dbf61c, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=07cef180b05347ebbcc6f3a6e97dac3a) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:58.394044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:58.395485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a3882-551d-4883-bf7e-1cf1b9f54aa3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:58.402689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 547/1105] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:58 2022] GET /identity/v3/role_assignments?user.id=07cef180b05347ebbcc6f3a6e97dac3a&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=2ca8c476a62e417c86da67ab92dbf61c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:53:58.446090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:58.481848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109c4d-3a30-40f3-9d51-5edb0ae7c9c7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:58.519123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:58.519733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:58.520009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:58.520276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:58.521046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=5c837b44ffb7450491045945971ed005, user.id=9ca7b16d991446808d84a12480134e77) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:58.521299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:58.522766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3f3e2-4820-4b03-bbd0-844e90ec46ae None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:58.529174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 559/1106] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:53:58 2022] GET /identity/v3/role_assignments?user.id=9ca7b16d991446808d84a12480134e77&scope.project.id=5c837b44ffb7450491045945971ed005&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:54:00.558786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefc4c8-b75e-4f0b-ae5b-bf191471c7e6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:00.559326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefc4c8-b75e-4f0b-ae5b-bf191471c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:00.559326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefc4c8-b75e-4f0b-ae5b-bf191471c7e6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:00.560661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 548/1107] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:00.566077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:00.566489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:00.566615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:00.585596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:00.586801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:00.652163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 560/1108] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:00.675960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefc4c8-b75e-4f0b-ae5b-bf191471c7e6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:00.774744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefc4c8-b75e-4f0b-ae5b-bf191471c7e6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:00.807219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f431eb-5565-439d-b10f-f502cf2d0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:00.807829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f431eb-5565-439d-b10f-f502cf2d0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:00.808103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f431eb-5565-439d-b10f-f502cf2d0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:00.808369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f431eb-5565-439d-b10f-f502cf2d0aa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:00.809456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f431eb-5565-439d-b10f-f502cf2d0aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:00.810890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f431eb-5565-439d-b10f-f502cf2d0aa7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:00.812436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 549/1109] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:54:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:00.818171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:00.876733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:00.912809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff44cd-63de-402a-b73a-8581ba094732 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:00.946630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1aff-a727-4d78-8238-76d81d189024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:00.947167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1aff-a727-4d78-8238-76d81d189024 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:00.947441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1aff-a727-4d78-8238-76d81d189024 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:00.947714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1aff-a727-4d78-8238-76d81d189024 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:00.948911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1aff-a727-4d78-8238-76d81d189024 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:00.950314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1aff-a727-4d78-8238-76d81d189024 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:01.089716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 561/1110] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:54:00 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:54:03.136896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12b3e7-3a70-406c-bf91-5f2659c16897 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:03.137471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12b3e7-3a70-406c-bf91-5f2659c16897 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:03.137471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12b3e7-3a70-406c-bf91-5f2659c16897 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:03.138840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 550/1111] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:03.144154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:03.144606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:03.144730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:03.162907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:03.164087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:03.228319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 562/1112] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:03.251472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12b3e7-3a70-406c-bf91-5f2659c16897 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:03.298858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:03.351082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12b3e7-3a70-406c-bf91-5f2659c16897 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:03.382286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:03.382909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:03.383185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:03.383451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] PATH_INFO: `/v3/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:03.384809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] RBAC: Authorizing `identity:get_role(role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:03.386715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:03.388057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 551/1113] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:03 2022] GET /identity/v3/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:03.394035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:03.450843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:03.491370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee0fe3-554c-4bb5-85e6-9304d05fa241 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:03.524044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:03.524662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:03.524942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:03.525214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] PATH_INFO: `/v3/users/328b260b2c684464bf7aba3e67ddea82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:03.538367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] RBAC: Authorizing `identity:get_user(user_id=328b260b2c684464bf7aba3e67ddea82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:03.539770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:03.541366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 563/1114] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:03 2022] GET /identity/v3/users/328b260b2c684464bf7aba3e67ddea82 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:03.547293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:03.642984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b324936-2db3-4769-ae94-bf6ae55e20d5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:03.673971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:03.674552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:03.674862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:03.675133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:03.738487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:03.739949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:03.749463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:03.835726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 552/1115] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:03.842253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:03.939836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b535619-ef92-4184-a95f-01fadb8004ac None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:03.946815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:03.972483 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a533113-ef39-4914-b3d2-c613ba8e45a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:03.973100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a533113-ef39-4914-b3d2-c613ba8e45a8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:03.973373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a533113-ef39-4914-b3d2-c613ba8e45a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:03.973639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a533113-ef39-4914-b3d2-c613ba8e45a8 None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:03.974841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a533113-ef39-4914-b3d2-c613ba8e45a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5c837b44ffb7450491045945971ed005)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:03.976245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a533113-ef39-4914-b3d2-c613ba8e45a8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:03.977837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 564/1116] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:03 2022] GET /identity/v3/projects/5c837b44ffb7450491045945971ed005 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:03.984716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:04.000035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:04.083003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5dfff-efd1-425c-b54b-ade5f4a96f10 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:04.116001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:04.116586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:04.116859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:04.117123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:04.117818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] RBAC: Adding query filter params (role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=5c837b44ffb7450491045945971ed005, user.id=328b260b2c684464bf7aba3e67ddea82) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:04.118118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:04.119557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f39c2-b459-45e1-ab52-aed9df563426 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:04.125489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 553/1117] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:54:03 2022] GET /identity/v3/role_assignments?user.id=328b260b2c684464bf7aba3e67ddea82&scope.project.id=5c837b44ffb7450491045945971ed005&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:06.097293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adfeb0-618c-41db-a72f-cfe9a6216292 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:06.097841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adfeb0-618c-41db-a72f-cfe9a6216292 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:06.097841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adfeb0-618c-41db-a72f-cfe9a6216292 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:06.099235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 565/1118] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:06.104406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:06.104821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:06.104941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:06.123153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:06.124315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:06.188735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 554/1119] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:06.210350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adfeb0-618c-41db-a72f-cfe9a6216292 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:06.308608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adfeb0-618c-41db-a72f-cfe9a6216292 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:06.342087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:06.342724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:06.343000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:06.343265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] PATH_INFO: `/v3/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:06.344627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] RBAC: Authorizing `identity:get_role(role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:06.345970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:06.347346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 566/1120] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:06 2022] GET /identity/v3/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:06.353173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:06.363593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:06.461341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5f34-7d44-4b87-a9dc-b25ddbf089d3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:06.494720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:06.495335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:06.495610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:06.495878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] PATH_INFO: `/v3/users/328b260b2c684464bf7aba3e67ddea82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:06.496975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] RBAC: Authorizing `identity:get_user(user_id=328b260b2c684464bf7aba3e67ddea82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:06.498662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:06.500237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 555/1121] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:06 2022] GET /identity/v3/users/328b260b2c684464bf7aba3e67ddea82 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:06.506016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:06.548234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:06.602032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80d30-5220-4f65-b5d9-95bd18b82ae1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:06.639563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27414-e776-494d-b3e9-5b965f88dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:06.640175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27414-e776-494d-b3e9-5b965f88dfcf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:06.640451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27414-e776-494d-b3e9-5b965f88dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:06.640720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27414-e776-494d-b3e9-5b965f88dfcf None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:06.641815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27414-e776-494d-b3e9-5b965f88dfcf None admin] RBAC: Authorizing `identity:get_project(project_id=5c837b44ffb7450491045945971ed005)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:06.643269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27414-e776-494d-b3e9-5b965f88dfcf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:06.644866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 567/1122] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:06 2022] GET /identity/v3/projects/5c837b44ffb7450491045945971ed005 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:06.650818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:06.733384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:06.757932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bc47f-bcaa-4fa9-ac45-296077e9685a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:06.791141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fedb4-c81d-4bdc-a138-83820ec69e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:06.791802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fedb4-c81d-4bdc-a138-83820ec69e4c None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:06.792075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fedb4-c81d-4bdc-a138-83820ec69e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:06.792345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fedb4-c81d-4bdc-a138-83820ec69e4c None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005/users/328b260b2c684464bf7aba3e67ddea82/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:06.794427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fedb4-c81d-4bdc-a138-83820ec69e4c None admin] RBAC: Authorizing `identity:create_grant(project_id=5c837b44ffb7450491045945971ed005, user_id=328b260b2c684464bf7aba3e67ddea82, role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:06.795905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fedb4-c81d-4bdc-a138-83820ec69e4c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:06.999436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 556/1123] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:54:06 2022] PUT /identity/v3/projects/5c837b44ffb7450491045945971ed005/users/328b260b2c684464bf7aba3e67ddea82/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:54:08.989601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5600301-eddc-4df0-8a03-c1d1279e82e2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:08.990154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5600301-eddc-4df0-8a03-c1d1279e82e2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:08.990154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5600301-eddc-4df0-8a03-c1d1279e82e2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:08.991657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 568/1124] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:08.997626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:08.998058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:08.998160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:09.016246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:09.017407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:09.080751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 557/1125] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:09.104152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5600301-eddc-4df0-8a03-c1d1279e82e2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:09.202387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5600301-eddc-4df0-8a03-c1d1279e82e2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:09.212575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:09.235153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:09.235768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:09.236045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:09.236314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] PATH_INFO: `/v3/roles/88effcf88ab94bc3af46d9e7b685ed0b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:09.237691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] RBAC: Authorizing `identity:get_role(role_id=88effcf88ab94bc3af46d9e7b685ed0b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:09.239075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:09.240445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 569/1126] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:09 2022] GET /identity/v3/roles/88effcf88ab94bc3af46d9e7b685ed0b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:09.246915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:09.309825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:09.342207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328469e2-5904-4cf9-a1a0-215109c6d079 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:09.376823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:09.377443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:09.377719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:09.377987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] PATH_INFO: `/v3/users/328b260b2c684464bf7aba3e67ddea82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:09.379120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] RBAC: Authorizing `identity:get_user(user_id=328b260b2c684464bf7aba3e67ddea82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:09.380537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:09.382106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 558/1127] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:09 2022] GET /identity/v3/users/328b260b2c684464bf7aba3e67ddea82 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:09.388151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:09.492957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9d02-35a2-4e82-8f50-28a41ea0ddb7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:09.526229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:09.526849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:09.527130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:09.527400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:09.571233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:09.592559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:09.593985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:09.688270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 570/1128] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:09.694198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:09.789800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708243f0-9b3d-4c69-aa85-6251833ffd2d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:09.822007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7acabc-402c-4ae0-8a36-885e17c2194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:09.823189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7acabc-402c-4ae0-8a36-885e17c2194e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:09.823617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7acabc-402c-4ae0-8a36-885e17c2194e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:09.823898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7acabc-402c-4ae0-8a36-885e17c2194e None admin] PATH_INFO: `/v3/projects/5c837b44ffb7450491045945971ed005` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:09.824989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7acabc-402c-4ae0-8a36-885e17c2194e None admin] RBAC: Authorizing `identity:get_project(project_id=5c837b44ffb7450491045945971ed005)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:09.826409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7acabc-402c-4ae0-8a36-885e17c2194e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:09.828028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 559/1129] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:09 2022] GET /identity/v3/projects/5c837b44ffb7450491045945971ed005 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:09.834163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:09.909041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:09.930287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54605e-8a1a-4c34-b2c8-695713e07d6a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:09.963242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:09.963834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:09.964110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:09.964379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:09.965089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] RBAC: Adding query filter params (role.id=88effcf88ab94bc3af46d9e7b685ed0b, scope.project.id=5c837b44ffb7450491045945971ed005, user.id=328b260b2c684464bf7aba3e67ddea82) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:09.965396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:09.966869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5492411-4aa3-4486-973f-7df1142c76c5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:09.972460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 571/1130] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:54:09 2022] GET /identity/v3/role_assignments?user.id=328b260b2c684464bf7aba3e67ddea82&scope.project.id=5c837b44ffb7450491045945971ed005&role.id=88effcf88ab94bc3af46d9e7b685ed0b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:11.990514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2519c-7241-4d0f-a644-1a3a28de718e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:11.991140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2519c-7241-4d0f-a644-1a3a28de718e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:11.991140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2519c-7241-4d0f-a644-1a3a28de718e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:11.992407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 560/1131] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:11.998306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:11.998993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:11.998993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:12.017631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:12.018858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:12.083295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 572/1132] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:12.105992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2519c-7241-4d0f-a644-1a3a28de718e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:12.108903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:12.204165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2519c-7241-4d0f-a644-1a3a28de718e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:12.236662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651190e-f400-4c5d-ba39-68b1d7d0321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:12.237269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651190e-f400-4c5d-ba39-68b1d7d0321e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:12.237546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651190e-f400-4c5d-ba39-68b1d7d0321e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:12.237815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651190e-f400-4c5d-ba39-68b1d7d0321e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:12.238932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651190e-f400-4c5d-ba39-68b1d7d0321e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:12.240357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651190e-f400-4c5d-ba39-68b1d7d0321e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:12.241939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 561/1133] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:54:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:12.248599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:12.315972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:12.343321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe776b-92d1-402d-b243-00a4214e3a7f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:12.376657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5716ac-0d56-4628-807f-1e90e5e537a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:12.377246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5716ac-0d56-4628-807f-1e90e5e537a2 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:12.377521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5716ac-0d56-4628-807f-1e90e5e537a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:12.377790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5716ac-0d56-4628-807f-1e90e5e537a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:12.378420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5716ac-0d56-4628-807f-1e90e5e537a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:12.379880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5716ac-0d56-4628-807f-1e90e5e537a2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:12.689802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 573/1134] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:54:12 2022] POST /identity/v3/projects => generated 311 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:54:14.716077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e470a41-2ade-4e93-b570-00d68bc8347a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:14.716607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e470a41-2ade-4e93-b570-00d68bc8347a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:14.716607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e470a41-2ade-4e93-b570-00d68bc8347a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:14.717859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 562/1135] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:14.722897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:14.723338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:14.723455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:14.740988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:14.742164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:14.811047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 574/1136] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:14.835196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e470a41-2ade-4e93-b570-00d68bc8347a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:14.902675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:14.932862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e470a41-2ade-4e93-b570-00d68bc8347a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:14.966891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeb415-646d-430b-ab9c-5ef220d155bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:14.967456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeb415-646d-430b-ab9c-5ef220d155bc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:14.967728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeb415-646d-430b-ab9c-5ef220d155bc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:14.967991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeb415-646d-430b-ab9c-5ef220d155bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:14.969075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aeb415-646d-430b-ab9c-5ef220d155bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:14.970486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aeb415-646d-430b-ab9c-5ef220d155bc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:14.972058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 563/1137] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:54:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:14.978208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:15.032618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:15.075256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471cb3e-b93d-458f-995a-7b61c7d0e7d3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:15.111623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a8353-f405-493d-ae69-90db055826c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:15.112219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a8353-f405-493d-ae69-90db055826c6 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:15.112493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a8353-f405-493d-ae69-90db055826c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:15.112760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a8353-f405-493d-ae69-90db055826c6 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:15.113389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a8353-f405-493d-ae69-90db055826c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:15.114828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a8353-f405-493d-ae69-90db055826c6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:15.289712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 575/1138] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:54:14 2022] POST /identity/v3/users => generated 286 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:54:17.316032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfcd20-e44d-46d0-a068-9813004fe0f9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:17.316595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfcd20-e44d-46d0-a068-9813004fe0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:17.316595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfcd20-e44d-46d0-a068-9813004fe0f9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:17.317905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 564/1139] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:17.323101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:17.323539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:17.323637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:17.341728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:17.342946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:17.412274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 576/1140] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:17.438829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfcd20-e44d-46d0-a068-9813004fe0f9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:17.537434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfcd20-e44d-46d0-a068-9813004fe0f9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:17.570807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:17.571346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:17.571621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:17.571886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:17.578363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:17.582035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:54:17.586702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:17.588026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:54:17.588990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 565/1141] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:54:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:54:17.594122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:17.634980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:17.690326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8243-4f31-4163-b7c9-cd537682b929 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:17.722719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:17.724115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:17.724115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:17.724115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:17.724486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:17.724786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:17.726143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:17.734374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 577/1142] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:54:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:17.740357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:17.852823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc7d85-b39d-478f-a56c-889f0e256020 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:17.885690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:17.886295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:17.886569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:17.886949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] PATH_INFO: `/v3/users/fc086e8c1b164bf8a887d0f01f35ad1d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:17.899340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] RBAC: Authorizing `identity:get_user(user_id=fc086e8c1b164bf8a887d0f01f35ad1d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:17.901287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:17.902893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 566/1143] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:17 2022] GET /identity/v3/users/fc086e8c1b164bf8a887d0f01f35ad1d => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:17.908758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:18.006405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb5ef-697b-4457-ada3-9b5f18b160a7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:18.007951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:18.039183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:18.039777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:18.040050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:18.040310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:18.105579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:18.106964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:18.213647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 578/1144] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:18.219727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:18.274346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:18.316169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6d0c-84f9-43cc-939f-bf6634dd74b9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:18.348919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a289e-40ba-46b1-b73b-66253b2b4595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:18.349519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a289e-40ba-46b1-b73b-66253b2b4595 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:18.349768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a289e-40ba-46b1-b73b-66253b2b4595 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:18.350033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a289e-40ba-46b1-b73b-66253b2b4595 None admin] PATH_INFO: `/v3/projects/6021809ad7e34944b0b9beea0e557e1e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:18.351160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a289e-40ba-46b1-b73b-66253b2b4595 None admin] RBAC: Authorizing `identity:get_project(project_id=6021809ad7e34944b0b9beea0e557e1e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:18.352572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a289e-40ba-46b1-b73b-66253b2b4595 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:18.354698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 567/1145] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:18 2022] GET /identity/v3/projects/6021809ad7e34944b0b9beea0e557e1e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:18.360847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:18.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:18.462925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acf8ab-21ee-4a55-a633-21bf62e54152 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:18.498940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:18.499538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:18.499813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:18.500082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:18.500789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=6021809ad7e34944b0b9beea0e557e1e, user.id=fc086e8c1b164bf8a887d0f01f35ad1d) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:18.501090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:18.502502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad382e8-a75f-4c52-abfb-5949b913e46c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:18.507864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 579/1146] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:54:18 2022] GET /identity/v3/role_assignments?user.id=fc086e8c1b164bf8a887d0f01f35ad1d&scope.project.id=6021809ad7e34944b0b9beea0e557e1e&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:20.468506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970542a2-09f9-4c21-aca7-b61e5c942bdf None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:20.468993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970542a2-09f9-4c21-aca7-b61e5c942bdf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:20.469114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970542a2-09f9-4c21-aca7-b61e5c942bdf None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:20.470348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 568/1147] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:20.475658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:20.476083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:20.476193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:20.494819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:20.495991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:20.553522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:20.561244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 580/1148] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:20.585028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970542a2-09f9-4c21-aca7-b61e5c942bdf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:20.684534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970542a2-09f9-4c21-aca7-b61e5c942bdf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:20.706513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:20.716501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:20.717109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:20.717383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:20.717649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:20.727234 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:54:20.732351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:20.733467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:54:20.734337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 569/1149] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:54:20.739558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:20.835260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd4c1e-53f7-42e8-8e94-c276c6a23ff2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:20.868345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:20.868971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:20.869248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:20.869515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:20.870177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:20.870477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:20.871859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:20.879852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 581/1150] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:20.885591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:20.982310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3b342-684b-4582-99d0-121097f982eb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:21.015169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:21.015767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:21.016038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:21.016301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] PATH_INFO: `/v3/users/fc086e8c1b164bf8a887d0f01f35ad1d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:21.017414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] RBAC: Authorizing `identity:get_user(user_id=fc086e8c1b164bf8a887d0f01f35ad1d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:21.018847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:21.020852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 570/1151] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:20 2022] GET /identity/v3/users/fc086e8c1b164bf8a887d0f01f35ad1d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:21.027119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:21.084681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:21.126455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f7c1-e7e1-4766-8c2e-6ff5101ed1c1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:21.159703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395f898-6c81-4af4-bfc0-3b630ca6b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:21.160317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f898-6c81-4af4-bfc0-3b630ca6b0fd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:21.160663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f898-6c81-4af4-bfc0-3b630ca6b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:21.160927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f898-6c81-4af4-bfc0-3b630ca6b0fd None admin] PATH_INFO: `/v3/projects/6021809ad7e34944b0b9beea0e557e1e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:21.162013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395f898-6c81-4af4-bfc0-3b630ca6b0fd None admin] RBAC: Authorizing `identity:get_project(project_id=6021809ad7e34944b0b9beea0e557e1e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:21.163440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395f898-6c81-4af4-bfc0-3b630ca6b0fd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:21.165036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 582/1152] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:21 2022] GET /identity/v3/projects/6021809ad7e34944b0b9beea0e557e1e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:21.171489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:21.222088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:21.267379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5efe3-919f-4b3d-924d-431c5f0ed478 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:21.304942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecd475-79a7-4de6-bb3f-49a8632682ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:21.305568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecd475-79a7-4de6-bb3f-49a8632682ff None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:21.305843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecd475-79a7-4de6-bb3f-49a8632682ff None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:21.306137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecd475-79a7-4de6-bb3f-49a8632682ff None admin] PATH_INFO: `/v3/projects/6021809ad7e34944b0b9beea0e557e1e/users/fc086e8c1b164bf8a887d0f01f35ad1d/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:21.308574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecd475-79a7-4de6-bb3f-49a8632682ff None admin] RBAC: Authorizing `identity:create_grant(project_id=6021809ad7e34944b0b9beea0e557e1e, user_id=fc086e8c1b164bf8a887d0f01f35ad1d, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:21.309990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecd475-79a7-4de6-bb3f-49a8632682ff None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:21.487869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 571/1153] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:54:21 2022] PUT /identity/v3/projects/6021809ad7e34944b0b9beea0e557e1e/users/fc086e8c1b164bf8a887d0f01f35ad1d/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:54:23.484536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929da00-bc41-4f3a-81e9-778228884be1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:23.485043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929da00-bc41-4f3a-81e9-778228884be1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:23.485043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929da00-bc41-4f3a-81e9-778228884be1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:23.486384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 583/1154] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:23.491696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a204b-3201-434d-a222-79f32759dec6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:23.492121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a204b-3201-434d-a222-79f32759dec6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:23.492227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a204b-3201-434d-a222-79f32759dec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:23.511284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-145a204b-3201-434d-a222-79f32759dec6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:23.512460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-145a204b-3201-434d-a222-79f32759dec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:23.576456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 572/1155] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:23.599243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929da00-bc41-4f3a-81e9-778228884be1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:23.699664 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929da00-bc41-4f3a-81e9-778228884be1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:23.724078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:23.732503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:23.733187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:23.733386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:23.733650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:23.744358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:54:23.749305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:23.750433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:54:23.751607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 584/1156] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:54:23.756579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a204b-3201-434d-a222-79f32759dec6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:23.798275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:23.851914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a204b-3201-434d-a222-79f32759dec6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:23.885139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:23.885723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:23.885997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:23.886267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:23.886944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:23.887242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:23.888596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:23.896280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 573/1157] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:23.902532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:23.997766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0079c-a1f6-47b1-95bd-253ddab3c2f8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:24.030072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:24.030689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:24.030969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:24.031234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] PATH_INFO: `/v3/users/fc086e8c1b164bf8a887d0f01f35ad1d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:24.032324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] RBAC: Authorizing `identity:get_user(user_id=fc086e8c1b164bf8a887d0f01f35ad1d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:24.033734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:24.035304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 585/1158] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:23 2022] GET /identity/v3/users/fc086e8c1b164bf8a887d0f01f35ad1d => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:24.041612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:24.139847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f00a0-175f-46fd-aed6-ea9ad7d78726 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:24.142079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:24.172331 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:24.172922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:24.173199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:24.173472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:24.237746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:24.239222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:24.333516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 574/1159] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:24.339419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:24.393052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:24.436118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a234b-00ec-44dc-9330-c52651079b81 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:24.468334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2af1a-a5fb-4cde-9719-8cc1ced6ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:24.468944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2af1a-a5fb-4cde-9719-8cc1ced6ab2b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:24.469219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2af1a-a5fb-4cde-9719-8cc1ced6ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:24.469487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2af1a-a5fb-4cde-9719-8cc1ced6ab2b None admin] PATH_INFO: `/v3/projects/6021809ad7e34944b0b9beea0e557e1e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:24.470579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2af1a-a5fb-4cde-9719-8cc1ced6ab2b None admin] RBAC: Authorizing `identity:get_project(project_id=6021809ad7e34944b0b9beea0e557e1e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:24.472021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2af1a-a5fb-4cde-9719-8cc1ced6ab2b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:24.473630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 586/1160] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:24 2022] GET /identity/v3/projects/6021809ad7e34944b0b9beea0e557e1e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:24.480280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:24.483367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:24.577372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ea6e7-2dc6-450e-8967-d326aee44adf None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:24.610867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:24.611390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:24.611673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:24.611940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:24.612647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=6021809ad7e34944b0b9beea0e557e1e, user.id=fc086e8c1b164bf8a887d0f01f35ad1d) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:24.612952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:24.614369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f548465-4ed3-42d3-938d-8cbdb48e6f8e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:24.625213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 575/1161] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:54:24 2022] GET /identity/v3/role_assignments?user.id=fc086e8c1b164bf8a887d0f01f35ad1d&scope.project.id=6021809ad7e34944b0b9beea0e557e1e&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:26.626338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59a253-1629-46e5-90f9-579d4afc33a2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:26.626886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59a253-1629-46e5-90f9-579d4afc33a2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:26.627046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59a253-1629-46e5-90f9-579d4afc33a2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:26.628393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 587/1162] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:26.633585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:26.633981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:26.634133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:26.655209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:26.656395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:26.716458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:26.721621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 576/1163] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:26.744761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59a253-1629-46e5-90f9-579d4afc33a2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:26.843099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59a253-1629-46e5-90f9-579d4afc33a2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:26.864591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:26.875526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:26.879105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:26.879105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:26.879105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:26.942550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:26.944024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:27.039305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 588/1164] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:27.045696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:27.140278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8644688-4b48-4828-89e5-18eb9fbd1284 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:27.173196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:27.173799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:27.174072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:27.174339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:27.180719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:27.182163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-dc8cd683-ec8f-4162-8775-ff373bf8366b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:27.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:27.194007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:54:27.194980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 577/1165] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:54:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:54:27.200616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:27.227553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:27.300127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfba58-6529-4718-bd71-3b14eab8ceca None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:27.332873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:27.333456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:27.333728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:27.333992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:27.334614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:27.334958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:27.336306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee31fa3-36c5-4438-bc0d-d12a1dbba9a5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:27.343139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 589/1166] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 10 22:54:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:29.334554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce7939-70cb-4c30-b65c-0ba15171769d None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:29.335117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce7939-70cb-4c30-b65c-0ba15171769d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:29.335117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce7939-70cb-4c30-b65c-0ba15171769d None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:29.336418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 578/1167] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:29.341619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cba9-1449-4807-90b6-f2b058969bd8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:29.341986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cba9-1449-4807-90b6-f2b058969bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:29.342170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cba9-1449-4807-90b6-f2b058969bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:29.360038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-eb11cba9-1449-4807-90b6-f2b058969bd8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:29.361224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eb11cba9-1449-4807-90b6-f2b058969bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:29.426825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 590/1168] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:29.449971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce7939-70cb-4c30-b65c-0ba15171769d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:29.507673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:29.547586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce7939-70cb-4c30-b65c-0ba15171769d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:29.580299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c3ef9-e6cb-48e6-b83b-2fd7ee5b7bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:29.580878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c3ef9-e6cb-48e6-b83b-2fd7ee5b7bfc None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:29.581157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c3ef9-e6cb-48e6-b83b-2fd7ee5b7bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:29.581424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c3ef9-e6cb-48e6-b83b-2fd7ee5b7bfc None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:29.582145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c3ef9-e6cb-48e6-b83b-2fd7ee5b7bfc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:29.583532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c3ef9-e6cb-48e6-b83b-2fd7ee5b7bfc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:29.690992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 579/1169] 10.222.0.78 () {60 vars in 1138 bytes} [Wed Aug 10 22:54:29 2022] POST /identity/v3/domains => generated 267 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:54:31.689947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ee39a-1feb-49a0-bd0b-e9456ecc81a3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:31.690452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ee39a-1feb-49a0-bd0b-e9456ecc81a3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:31.690452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ee39a-1feb-49a0-bd0b-e9456ecc81a3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:31.691766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 591/1170] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:31.697037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:31.697507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:31.697606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:31.715645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:31.716837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:31.790224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 580/1171] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:31.812908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ee39a-1feb-49a0-bd0b-e9456ecc81a3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:31.831490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:31.914449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ee39a-1feb-49a0-bd0b-e9456ecc81a3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:31.948006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24aa32-75d7-48c9-901f-e379248dd39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:31.948550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24aa32-75d7-48c9-901f-e379248dd39f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:31.948824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24aa32-75d7-48c9-901f-e379248dd39f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:31.949092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24aa32-75d7-48c9-901f-e379248dd39f None admin] PATH_INFO: `/v3/domains/a0fb8a06cd5545879a2abd5fb3f30899` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:31.959485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24aa32-75d7-48c9-901f-e379248dd39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=a0fb8a06cd5545879a2abd5fb3f30899)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:31.960916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24aa32-75d7-48c9-901f-e379248dd39f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:31.963099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 592/1172] 10.222.0.78 () {56 vars in 1183 bytes} [Wed Aug 10 22:54:31 2022] GET /identity/v3/domains/a0fb8a06cd5545879a2abd5fb3f30899 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:31.968953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:32.017985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:32.069035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dad89-b4ff-436c-a3f1-872023e2dacd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:32.105276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f0da9-de1a-4aa2-b0ed-29d18521960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:32.106035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f0da9-de1a-4aa2-b0ed-29d18521960d None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:32.106309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f0da9-de1a-4aa2-b0ed-29d18521960d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:32.106572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f0da9-de1a-4aa2-b0ed-29d18521960d None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:32.107227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f0da9-de1a-4aa2-b0ed-29d18521960d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:32.108639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f0da9-de1a-4aa2-b0ed-29d18521960d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:32.268353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 581/1173] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:54:31 2022] POST /identity/v3/projects => generated 361 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:54:34.347219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287db0-a4b0-4f20-80b8-8b1961561e2b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:34.347800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287db0-a4b0-4f20-80b8-8b1961561e2b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:34.347800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287db0-a4b0-4f20-80b8-8b1961561e2b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:34.349077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 593/1174] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:34.354673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:34.355118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:34.355214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:34.372971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:34.374131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:34.440593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 582/1175] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:34.464743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287db0-a4b0-4f20-80b8-8b1961561e2b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:34.539778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:34.563330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287db0-a4b0-4f20-80b8-8b1961561e2b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:34.595609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd3e92-d947-4072-9cb7-38e566b8b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:34.596219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd3e92-d947-4072-9cb7-38e566b8b9cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:34.596495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd3e92-d947-4072-9cb7-38e566b8b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:34.596763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd3e92-d947-4072-9cb7-38e566b8b9cf None admin] PATH_INFO: `/v3/domains/a0fb8a06cd5545879a2abd5fb3f30899` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:34.597890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd3e92-d947-4072-9cb7-38e566b8b9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=a0fb8a06cd5545879a2abd5fb3f30899)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:34.599290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd3e92-d947-4072-9cb7-38e566b8b9cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:34.600817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 594/1176] 10.222.0.78 () {56 vars in 1183 bytes} [Wed Aug 10 22:54:34 2022] GET /identity/v3/domains/a0fb8a06cd5545879a2abd5fb3f30899 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:34.606553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:34.635951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:34.702024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8137ff9-7378-46f9-bcac-420e5c172cc2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:34.736566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48754cd8-aaa7-4f79-a8d0-4dc18f71ee35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:34.737166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48754cd8-aaa7-4f79-a8d0-4dc18f71ee35 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:34.737450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48754cd8-aaa7-4f79-a8d0-4dc18f71ee35 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:34.737722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48754cd8-aaa7-4f79-a8d0-4dc18f71ee35 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:34.738368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48754cd8-aaa7-4f79-a8d0-4dc18f71ee35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:34.739841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48754cd8-aaa7-4f79-a8d0-4dc18f71ee35 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:34.870827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 583/1177] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:54:34 2022] POST /identity/v3/users => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:54:36.908215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eade22-cb69-468e-b468-2267b1bb9a5c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:36.908734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eade22-cb69-468e-b468-2267b1bb9a5c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:36.908848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eade22-cb69-468e-b468-2267b1bb9a5c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:36.910095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 595/1178] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:36.915771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:36.916144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:36.916304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:36.934587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:36.935778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:36.999346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 584/1179] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:37.022202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eade22-cb69-468e-b468-2267b1bb9a5c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:37.121042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eade22-cb69-468e-b468-2267b1bb9a5c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:37.152958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:37.153560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:37.153832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:37.154094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:37.164399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:54:37.169585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:37.170740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:54:37.171813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 596/1180] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:54:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:54:37.176840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:37.194355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:37.272740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490067b-aae0-4642-a270-d0e4b46aaab8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:37.306145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:37.306759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:37.307037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:37.307304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:37.307953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:37.308249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:37.309594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:37.318011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 585/1181] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:54:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:37.324390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:37.328132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:37.426907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1daae-ece0-469a-870b-70e41fb8a8eb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:37.459839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:37.460449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:37.460721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:37.460987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] PATH_INFO: `/v3/users/8c4115ea2355425dba6693a5623b5668` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:37.473435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] RBAC: Authorizing `identity:get_user(user_id=8c4115ea2355425dba6693a5623b5668)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:37.474878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:37.476451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 597/1182] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:37 2022] GET /identity/v3/users/8c4115ea2355425dba6693a5623b5668 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:37.482345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:37.559188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:37.579957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f444d-a5f9-4f52-a67c-e536c837549c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:37.611612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:37.612196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:37.612472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:37.612740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:37.679491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:37.681585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:37.757460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:37.778158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 586/1183] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:37.784179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:37.843970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:37.881410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591aba9-30f8-455c-aeaa-130965636a41 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:37.913688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ae90c-39b7-4737-84ae-a521fe75ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:37.914283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ae90c-39b7-4737-84ae-a521fe75ff24 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:37.915088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ae90c-39b7-4737-84ae-a521fe75ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:37.915369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ae90c-39b7-4737-84ae-a521fe75ff24 None admin] PATH_INFO: `/v3/projects/5a3a2aedd76842158d002df65dc490cc` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:37.916467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ae90c-39b7-4737-84ae-a521fe75ff24 None admin] RBAC: Authorizing `identity:get_project(project_id=5a3a2aedd76842158d002df65dc490cc)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:37.917877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ae90c-39b7-4737-84ae-a521fe75ff24 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:37.919527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 598/1184] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:37 2022] GET /identity/v3/projects/5a3a2aedd76842158d002df65dc490cc => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:37.925646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:38.028916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb2a6b-1a90-411f-aaf7-f27c15fb73c5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:38.063521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:38.064113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:38.064391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:38.064658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:38.065368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=5a3a2aedd76842158d002df65dc490cc, user.id=8c4115ea2355425dba6693a5623b5668) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:38.065682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:38.067131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ec960-50d9-4f9d-a7f6-f7015791ad4a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:38.072953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 587/1185] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:54:37 2022] GET /identity/v3/role_assignments?user.id=8c4115ea2355425dba6693a5623b5668&scope.project.id=5a3a2aedd76842158d002df65dc490cc&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:40.053092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee299-feea-43f3-9b4e-3021779cb1a1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.053657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee299-feea-43f3-9b4e-3021779cb1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.053657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee299-feea-43f3-9b4e-3021779cb1a1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.055001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 599/1186] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:40.060580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af70621-ac93-4894-866f-87549ff07725 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.060999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af70621-ac93-4894-866f-87549ff07725 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.061115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af70621-ac93-4894-866f-87549ff07725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.079105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3af70621-ac93-4894-866f-87549ff07725 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:40.080291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3af70621-ac93-4894-866f-87549ff07725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:40.113762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:40.144177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 588/1187] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:40.166549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee299-feea-43f3-9b4e-3021779cb1a1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:40.265680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee299-feea-43f3-9b4e-3021779cb1a1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:40.294081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:40.299374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:40.299988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.300262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.300532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.310243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:54:40.314828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:40.315964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:54:40.317046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 600/1188] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:54:40.322196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af70621-ac93-4894-866f-87549ff07725 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:40.418577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af70621-ac93-4894-866f-87549ff07725 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:40.452779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:40.453367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.453642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.453909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.454550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:40.455023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:40.456372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:40.464103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 589/1189] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:40.469908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:40.566311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8357e-cc64-4af4-9e5f-68d8594f20c7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:40.598889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:40.599500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.599772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.600036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] PATH_INFO: `/v3/users/8c4115ea2355425dba6693a5623b5668` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.601134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] RBAC: Authorizing `identity:get_user(user_id=8c4115ea2355425dba6693a5623b5668)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:40.602613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:40.604223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 601/1190] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:40 2022] GET /identity/v3/users/8c4115ea2355425dba6693a5623b5668 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:40.610175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:40.646814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:40.714139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b06ca-9140-45a6-b964-494db8a7116a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:40.750134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75826c59-2c8d-4b74-b7d5-668b1d3f017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:40.750772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75826c59-2c8d-4b74-b7d5-668b1d3f017a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.751047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75826c59-2c8d-4b74-b7d5-668b1d3f017a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.751315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75826c59-2c8d-4b74-b7d5-668b1d3f017a None admin] PATH_INFO: `/v3/projects/5a3a2aedd76842158d002df65dc490cc` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.752420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75826c59-2c8d-4b74-b7d5-668b1d3f017a None admin] RBAC: Authorizing `identity:get_project(project_id=5a3a2aedd76842158d002df65dc490cc)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:40.753938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75826c59-2c8d-4b74-b7d5-668b1d3f017a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:40.755593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 590/1191] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:40 2022] GET /identity/v3/projects/5a3a2aedd76842158d002df65dc490cc => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:40.765282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:40.825754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:40.867218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a1291-ff01-4c56-bf27-f268541d3523 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:40.903152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c951d7-c02a-4e2b-8316-b9037ebf0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:40.903804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c951d7-c02a-4e2b-8316-b9037ebf0de9 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:40.904084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c951d7-c02a-4e2b-8316-b9037ebf0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:40.904355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c951d7-c02a-4e2b-8316-b9037ebf0de9 None admin] PATH_INFO: `/v3/projects/5a3a2aedd76842158d002df65dc490cc/users/8c4115ea2355425dba6693a5623b5668/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:40.906247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c951d7-c02a-4e2b-8316-b9037ebf0de9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3a2aedd76842158d002df65dc490cc, user_id=8c4115ea2355425dba6693a5623b5668, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:40.907720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c951d7-c02a-4e2b-8316-b9037ebf0de9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:41.061463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 602/1192] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:54:40 2022] PUT /identity/v3/projects/5a3a2aedd76842158d002df65dc490cc/users/8c4115ea2355425dba6693a5623b5668/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:54:43.022003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae02f1-7139-411a-8888-f9d20df67958 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:43.022590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae02f1-7139-411a-8888-f9d20df67958 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:43.022590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae02f1-7139-411a-8888-f9d20df67958 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:43.023884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 591/1193] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:54:43.029222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:43.029622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:43.029768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:43.048785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:54:43.049966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:43.113962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 603/1194] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:54:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:54:43.137309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae02f1-7139-411a-8888-f9d20df67958 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:43.235189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae02f1-7139-411a-8888-f9d20df67958 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:43.236661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:43.270989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:43.271596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:43.271872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:43.272190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:43.282313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:54:43.286918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:54:43.288072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:54:43.289482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 592/1195] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:54:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:54:43.294144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:43.341958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:43.390269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33565e-d734-4e82-8c64-9f40e8c50e7f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:43.422555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:43.423185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:43.423465 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:43.423735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:43.424392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:43.425027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:43.426054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:43.434722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 604/1196] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:54:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:43.440501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:43.537375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af435c7-7640-4390-8813-afaf8e414756 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:43.569335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:43.569941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:43.570216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:43.570511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] PATH_INFO: `/v3/users/8c4115ea2355425dba6693a5623b5668` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:43.572863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] RBAC: Authorizing `identity:get_user(user_id=8c4115ea2355425dba6693a5623b5668)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:43.574276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:43.575864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 593/1197] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 10 22:54:43 2022] GET /identity/v3/users/8c4115ea2355425dba6693a5623b5668 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:43.581671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:43.678402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a227825-d4c8-4a11-bdbf-46bf0c67e218 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:43.689013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:43.711199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:43.711795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:43.712072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:43.712340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:43.781406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:43.782866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:43.877080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 605/1198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:54:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:54:43.883023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:43.913057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:43.978195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b77980-ca37-488a-b9e2-b8c41773c5cd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:44.011091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbcff3-1c0c-4027-abea-f1e402c2a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:44.011693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbcff3-1c0c-4027-abea-f1e402c2a18b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:44.011963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbcff3-1c0c-4027-abea-f1e402c2a18b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:44.012227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbcff3-1c0c-4027-abea-f1e402c2a18b None admin] PATH_INFO: `/v3/projects/5a3a2aedd76842158d002df65dc490cc` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:44.013309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbcff3-1c0c-4027-abea-f1e402c2a18b None admin] RBAC: Authorizing `identity:get_project(project_id=5a3a2aedd76842158d002df65dc490cc)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:44.015127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbcff3-1c0c-4027-abea-f1e402c2a18b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:44.020400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 594/1199] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 10 22:54:43 2022] GET /identity/v3/projects/5a3a2aedd76842158d002df65dc490cc => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:44.028271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:44.040025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:44.125406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1f6d-0662-44ec-84d6-d7a383561ae8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:44.158099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:44.158622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:44.158937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:44.159201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:44.159908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=5a3a2aedd76842158d002df65dc490cc, user.id=8c4115ea2355425dba6693a5623b5668) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:54:44.160208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:44.161627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a00f08-0dd4-48ad-811f-a866dae65e10 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:44.167204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 606/1200] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:54:44 2022] GET /identity/v3/role_assignments?user.id=8c4115ea2355425dba6693a5623b5668&scope.project.id=5a3a2aedd76842158d002df65dc490cc&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:54:54.006968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=51 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:54.164634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=63 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:12.741898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954d52-1777-4604-8b1e-a5721bcc9d5b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:12.742468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954d52-1777-4604-8b1e-a5721bcc9d5b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:12.742468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954d52-1777-4604-8b1e-a5721bcc9d5b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:12.743982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 595/1201] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:12.749042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:12.749481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:12.749582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:12.768614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:12.769860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:12.835836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 607/1202] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:12.859481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954d52-1777-4604-8b1e-a5721bcc9d5b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:12.961129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954d52-1777-4604-8b1e-a5721bcc9d5b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:12.994711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67034b8-b075-478b-827b-845604d2688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:12.995273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67034b8-b075-478b-827b-845604d2688f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:12.995551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67034b8-b075-478b-827b-845604d2688f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:12.995822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67034b8-b075-478b-827b-845604d2688f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:13.006253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67034b8-b075-478b-827b-845604d2688f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:13.007729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67034b8-b075-478b-827b-845604d2688f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d67034b8-b075-478b-827b-845604d2688f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:13.021384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:13.028450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:13.029780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:13.029780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:13.029780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:13.029780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 596/1203] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:13.030086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:13.125351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20293b3-12a7-4b6e-ae10-d149804bd159 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:13.152656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:13.158364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:13.159076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:13.159349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:13.159612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:13.160741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:13.161022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:13.162371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a4ec-2652-47df-bca5-8848743fdc02 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:13.173129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 608/1204] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:13.179468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67034b8-b075-478b-827b-845604d2688f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:13.232184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:13.278092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67034b8-b075-478b-827b-845604d2688f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:13.311216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfe10a-a666-4264-8c3a-9279dc799b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:13.311809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfe10a-a666-4264-8c3a-9279dc799b20 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:13.312083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfe10a-a666-4264-8c3a-9279dc799b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:13.312355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfe10a-a666-4264-8c3a-9279dc799b20 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:13.312989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfe10a-a666-4264-8c3a-9279dc799b20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:13.314408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfe10a-a666-4264-8c3a-9279dc799b20 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:13.554716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 597/1205] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 10 22:56:13 2022] POST /identity/v3/users => generated 251 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:56:15.600169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75863f3d-8d52-4d66-9f2a-4386eb441fc3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:15.600745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75863f3d-8d52-4d66-9f2a-4386eb441fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:15.600745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75863f3d-8d52-4d66-9f2a-4386eb441fc3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:15.602019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 609/1206] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:15.607213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:15.607689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:15.607790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:15.626422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:15.627644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:15.702195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 598/1207] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:15.725313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75863f3d-8d52-4d66-9f2a-4386eb441fc3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:15.826303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75863f3d-8d52-4d66-9f2a-4386eb441fc3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:15.858427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:15.860908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:15.860908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:15.860908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:15.870949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:56:15.875652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:15.876940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:15.878069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 610/1208] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:56:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:15.883372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:15.923238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:15.980737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c481a-3d25-406f-b3ab-c32800a36780 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.022914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.023519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.023789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.024059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.024708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:16.025133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.026496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.034318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 599/1209] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:16.040180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.090735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.139648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54413b-3455-4709-a1c7-be06033ff759 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.172535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.178122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.178122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.178122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.183728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.185149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:16.199186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:16.200280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:16.201743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:16.201743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:16.201743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:16.201743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 611/1210] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:16.206343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.303869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.306877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad21cc-3a1d-4959-8de7-5ffe9914ce4a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.340491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.341082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.341356 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.341622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.342283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:16.342742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.344622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.355533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 600/1211] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:16.361407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.460722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06a73-dd27-4cb6-b101-0b46da62f3c0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.462217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.493269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.494130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.494399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.494688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.501671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.503131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:56:16.515046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:16.521106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:16.522108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 612/1212] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:56:16.522885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.620424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5cda0-ff38-499a-8617-1449b1f3a941 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.653886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.654488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.654795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.655070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.655787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:16.656091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.657513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.674940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 601/1213] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 10 22:56:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:16.681374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.781471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0acd9b-30c7-49a3-9354-4a8784e00720 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.814667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.815266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.815538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.815807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.844230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.881317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.882683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.980023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 613/1214] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:56:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:56:16.985991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:17.005512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:17.084767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0a5a-9918-4862-bccd-144ee03b0a8f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:17.118067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:17.118963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:17.119240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:17.119547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:17.129494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:17.131012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:17.151896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:17.156876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:17.158073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:17.158073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:17.158073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:17.158073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 602/1215] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:17.158758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:17.222238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:17.260327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872e323-fa42-416a-9c56-9cefebf53bf1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:17.294433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:17.295063 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:17.295361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:17.295631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:17.296258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:17.296557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:17.297898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:17.309435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 614/1216] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:17.315320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:17.384886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:17.417225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf106-6568-469b-a798-e87c15ab6280 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:17.449990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:17.450603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:17.450924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:17.451201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:17.460566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:17.462055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:56:17.473051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:17.474213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:17.475469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 603/1217] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:56:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:17.480309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:17.579469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6852ee-4a37-490f-8fb8-86a0ff9538b7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:17.601461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:17.613030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:17.613614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:17.613889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:17.614156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:17.614860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:17.615162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:17.616576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d6cc-cdfe-49a1-a863-a0db74030e25 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:17.628651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 615/1218] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:56:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:17.635220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:17.752992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3dace-bba9-4eb0-9949-79f58133db8d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:17.783780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:17.789687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:17.790280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:17.790554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:17.790855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:17.791631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=2604f58c168e432d812608a2351610bd) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:17.791926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:17.793350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c3ffb-cf82-4b42-bf8c-93960710c3c6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:17.798526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 604/1219] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:56:17 2022] GET /identity/v3/role_assignments?user.id=2604f58c168e432d812608a2351610bd&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:19.772578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cca934-f2b2-4340-a5e8-ca790cc17c2b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:19.773122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cca934-f2b2-4340-a5e8-ca790cc17c2b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:19.773122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cca934-f2b2-4340-a5e8-ca790cc17c2b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:19.774675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 616/1220] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:19.779719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:19.780140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:19.780246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:19.797940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:19.799147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:19.864636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 605/1221] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:19.887436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cca934-f2b2-4340-a5e8-ca790cc17c2b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:19.990550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cca934-f2b2-4340-a5e8-ca790cc17c2b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.025296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.025848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.026123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.026390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.040601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:56:20.045363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:20.046495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:20.047614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 617/1222] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:56:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:20.053104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:20.150386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb867a-bbb5-43ae-8d6f-02f8cd9a8156 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.173162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:20.183851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.184443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.184718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.184986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.185640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:20.185936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:20.187328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:20.195673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 606/1223] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:20.201517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:20.245594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:20.303295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1a826-0803-4acc-880c-d42ee91c4c7a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.335837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.336453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.336727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.336995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.347083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:20.348537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:20.362978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:20.365317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:20.366880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:20.366880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:20.366880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:20.366880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 618/1224] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:20.370908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:20.471310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ecdd-d1d3-4251-84ba-3acba8c2949d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.503854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.506982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.506982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.506982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.506982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:20.506982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:20.507728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:20.518954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 607/1225] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:20.525319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:20.615926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:20.622490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e76d1-2ef3-4dac-b8f9-101d7455e820 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.662300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.662982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.668170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.668170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.671214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:20.672687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:56:20.684536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:20.693315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:20.694584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 619/1226] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:56:20.695235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:20.737470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:20.790503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31d4d2-6533-40e1-9772-c931f5589d85 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.824519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.825100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.825369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.825636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.826340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:20.826632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:20.828423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:20.845237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 608/1227] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 10 22:56:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:20.851578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:20.948308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c519c-5a55-43da-95a5-f693d73452ce None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:20.982088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:20.982789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:20.983099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:20.983339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:20.993976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:20.995439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:21.009117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:21.010340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:21.012382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:21.012382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:21.012382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:21.012382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 620/1228] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:21.016731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:21.120485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925d1a-6970-427f-bcb1-38644b4702f6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:21.122789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:21.154155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:21.154770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:21.155043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:21.155309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:21.155931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:21.156223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:21.157561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:21.168831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 609/1229] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:21.174518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:21.184241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:21.275799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5dac7-59ba-47c7-9811-15529bc2fd56 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:21.310442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:21.311428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:21.311428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:21.311691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:21.318138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:21.319611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:56:21.330362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:21.331521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:21.332659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 621/1230] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:21.337392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:21.443726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9066b-45b5-49cc-a975-73bf302878e2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:21.476795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:21.477391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:21.477664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:21.477932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:21.478603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:21.478939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:21.480355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b015b23-8a08-44ef-896b-dd1625220694 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:21.492639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 610/1231] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:21.499021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:21.564408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:21.597850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df89d-3b66-4ab8-844c-71398bb40f8f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:21.630417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c0b52-493f-4c3c-aff1-ef2fe0fd1abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:21.631092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c0b52-493f-4c3c-aff1-ef2fe0fd1abd None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:21.631368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c0b52-493f-4c3c-aff1-ef2fe0fd1abd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:21.631636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c0b52-493f-4c3c-aff1-ef2fe0fd1abd None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/2604f58c168e432d812608a2351610bd/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:21.645467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c0b52-493f-4c3c-aff1-ef2fe0fd1abd None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=2604f58c168e432d812608a2351610bd, role_id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:21.646914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c0b52-493f-4c3c-aff1-ef2fe0fd1abd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:21.830801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 622/1232] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:56:21 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/2604f58c168e432d812608a2351610bd/roles/ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:23.842430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8343b1-2a13-4246-9757-c01017de761d None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:23.843008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8343b1-2a13-4246-9757-c01017de761d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:23.843008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8343b1-2a13-4246-9757-c01017de761d None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:23.844385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 611/1233] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:23.849500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:23.849888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:23.850155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:23.868328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:23.869793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:23.939505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 623/1234] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:23.963332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8343b1-2a13-4246-9757-c01017de761d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:23.987859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:24.064081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8343b1-2a13-4246-9757-c01017de761d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:24.098056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:24.098639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:24.098943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:24.099216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:24.108963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:56:24.114089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:24.115239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:24.116262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 612/1235] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 10 22:56:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:24.121746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:24.168151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:24.218799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debffe2-7ffa-4ad8-a59d-babbf79990a5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:24.252263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:24.252901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:24.253178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:24.253446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:24.254119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:24.254420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:24.255803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:24.263365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 624/1236] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 10 22:56:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:56:24.269288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:24.351276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:24.366463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232840d-08bf-4e03-ab8c-39b3c5cbd70d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:24.400889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:24.401537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:24.401813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:24.402080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:24.412303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:24.413729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:24.430430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:24.435498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:24.436647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:24.436647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:24.436647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:24.436647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 613/1237] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:24.436978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:24.538320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76696-6b18-475a-a9e2-d4408a8779cf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:24.547035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:24.572612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:24.573217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:24.573493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:24.573763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:24.574393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:24.574711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:24.576069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:24.587496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 625/1238] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:24.593260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:24.692811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ab3d3-b04a-40a0-bbdc-37a18b85a03d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:24.725868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:24.726474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:24.726781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:24.727051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:24.729615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:24.734577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:24.736026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:56:24.747349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:24.755596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:24.756627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 614/1239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:24.757238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:24.854713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e188a5-5b31-41cc-a8ca-ce436f628104 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:24.889224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:24.889828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:24.890091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:24.890357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:24.891099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:24.891563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:24.895051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:24.912351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 626/1240] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 10 22:56:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:24.918208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:25.017620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0083b-d5da-4ed9-a9d9-13a0e9586aa7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:25.051700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:25.052335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:25.052613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:25.052885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:25.112756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:25.119984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:25.121437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:25.215704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 615/1241] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:56:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:56:25.221493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:25.249539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:25.321832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6b17-f218-4e68-97fc-42ff21d29ec2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:25.353208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:25.353816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:25.354089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:25.354351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:25.364829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:25.379689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:25.381344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:25.382536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:25.383738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:25.383738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:25.383738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:25.383738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 627/1242] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:25.388132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:25.485048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:25.492175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d1d8-4e78-4d51-bbe2-d05cbfb0057e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:25.525572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:25.526163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:25.526433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:25.526729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:25.527352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:25.527646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:25.528994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:25.539890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 616/1243] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:25.545635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:25.624403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:25.645244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70265cc2-68af-44b6-a0fd-74063d13602a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:25.678675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:25.679319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:25.679595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:25.679866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:25.686233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:25.687690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:56:25.698902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:25.700076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:25.701171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 628/1244] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:56:25.706102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:25.812706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d863-c743-4a70-ae43-7265c85eeb02 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:25.845414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:25.846000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:25.846269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:25.846531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:25.847230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:25.847524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:25.849450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be921d7-7447-4004-a7e4-e615bf112277 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:25.860930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 617/1245] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:25.867796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:25.969670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b424c4-cc82-4a49-9c47-23a49c9dc9d1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:26.003265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:26.003875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:26.004147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:26.004415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:26.005125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] RBAC: Adding query filter params (role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=2604f58c168e432d812608a2351610bd) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:26.005457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:26.006901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f3ba1-1989-42f4-85bd-3973749a0e49 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:26.012544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 629/1246] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:56:25 2022] GET /identity/v3/role_assignments?user.id=2604f58c168e432d812608a2351610bd&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=ce36b1a8ee9b4dc8b535e1f4ba4a44dd => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:28.068237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07d666-be1c-4d4b-8000-04b25b01fe7a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.068794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07d666-be1c-4d4b-8000-04b25b01fe7a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.068794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07d666-be1c-4d4b-8000-04b25b01fe7a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.070090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 618/1247] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:28.075336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.075786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.075906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.082092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:28.094310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:28.095535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:28.164408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 630/1248] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:28.188562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07d666-be1c-4d4b-8000-04b25b01fe7a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:28.208671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:28.297381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07d666-be1c-4d4b-8000-04b25b01fe7a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:28.334255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:28.334822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.335096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.335360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.346432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:56:28.352696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:28.359807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:28.360740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 619/1249] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:28.361639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:28.471146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1178a3-62d8-420e-9612-6468f15c9643 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:28.501497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:28.506918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:28.507517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.507792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.508058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.508714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:28.509013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:28.510404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:28.518240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 631/1250] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:28.524022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:28.605702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:28.628381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb589b-8c51-4a99-8c5a-18132119c4b7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:28.661348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:28.661957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.662235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.662505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.673091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:28.674537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:28.688662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:28.689843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:28.690778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:28.690778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:28.690778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:28.690778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 620/1251] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:28.695669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:28.795777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924d5d8-4a28-4c35-8345-d4d486e27622 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:28.828886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:28.829498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.829831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.830101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.830765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:28.831067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:28.832460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:28.843885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 632/1252] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:28.850168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:28.948410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66c5c6-1360-4678-b70e-24e02bf6f597 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:28.974848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:28.982466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:28.983114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:28.983392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:28.983660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:28.991816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:28.993241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:56:29.004791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:29.005961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:29.007042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 621/1253] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:29.012186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:29.070697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:29.113004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135236-2d88-4ce1-8a77-85a3ea8b1ced None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:29.146571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:29.147205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:29.147481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:29.147750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:29.148465 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:29.148768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:29.150243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:29.167830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 633/1254] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 10 22:56:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:29.172760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:29.271576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21250c5c-1ac1-4c6f-909e-ce2b4b1e70b4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:29.303867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:29.304461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:29.304736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:29.305003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:29.359239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:29.372298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:29.373754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:29.468013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 622/1255] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:56:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:56:29.474491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:29.573119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ccb5b-dcb0-4579-9210-168ce954c5ab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:29.585796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:29.606152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:29.606864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:29.607145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:29.607418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:29.617669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:29.619394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:29.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:29.634545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:29.635493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:29.635493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:29.635493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:29.635493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 634/1256] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:29.640368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:29.724300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:29.735739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835f0cd-ab73-4884-96a0-55121f941e17 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:29.768399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:29.768977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:29.769248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:29.770106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:29.770648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:29.770969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:29.772311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:29.782897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 623/1257] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:29.790023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:29.889595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daafc5-275c-497b-bf77-c3f5f7eb18dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:29.925655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:29.926320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:29.926592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:29.926884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:29.933382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:29.934839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:56:29.946356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:29.947575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:29.948782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 635/1258] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:29.953481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:30.053148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e405c6-707b-43fe-be50-f053c9e1336c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:30.085881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:30.086472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:30.086778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:30.087049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:30.087725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:30.088025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:30.089463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e835628-2ce9-4e10-9620-e6dbcacff3f5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:30.101209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 624/1259] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:30.107371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:30.133934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:30.210727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee70b2c-b627-47c8-93fe-56e279bb03c1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:30.244385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:30.244970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:30.245241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:30.245505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:30.246202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=2604f58c168e432d812608a2351610bd) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:30.246500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:30.247931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0d490-4b16-4a9e-bb21-4928dae2a4fe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:30.253077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 636/1260] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:56:30 2022] GET /identity/v3/role_assignments?user.id=2604f58c168e432d812608a2351610bd&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=be98a6e301f144d88c992f65bca561fb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:32.230710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e1d6-d653-4e0a-b437-024af4adc0b3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:32.231324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e1d6-d653-4e0a-b437-024af4adc0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:32.231324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e1d6-d653-4e0a-b437-024af4adc0b3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:32.232559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 625/1261] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:56:32.237693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:32.238083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:32.238279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:32.256187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:32.257395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:32.321756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 637/1262] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:32.344409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e1d6-d653-4e0a-b437-024af4adc0b3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:32.355712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:32.445250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e1d6-d653-4e0a-b437-024af4adc0b3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:32.479053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:32.479660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:32.479930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:32.481162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:32.490116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:56:32.495040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:32.496181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:32.497061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 626/1263] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:56:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:32.502310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:32.521297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:32.599006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b0aa0-bc0f-4bae-9cc2-ea18d3e402b7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:32.631936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:32.632531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:32.632807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:32.633075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:32.633725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:32.634024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:32.635530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:32.643273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 638/1264] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:56:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:32.649586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:32.715108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:32.747432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be49e-fe08-4117-b46c-c46b06513853 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:32.784121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:32.784748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:32.785025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:32.785314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:32.795307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:32.796810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:32.812411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:32.823498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:32.824666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:32.824666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:32.824666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:32.824666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 627/1265] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:32.824970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:32.900397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:32.918702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db4d15-fafa-4266-b1f0-a410bd970ef6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:32.951248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:32.951866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:32.952142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:32.952409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:32.953100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:32.953395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:32.954785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:32.966451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 639/1266] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:32.974363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:33.070393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a45f91-1e13-4fbb-9c8b-28380fcb40f9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:33.089123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:33.102217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:33.102908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:33.103180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:33.103440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:33.111421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:33.117859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:56:33.127533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:33.128720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:33.130265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 628/1267] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:33.135915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:33.234252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955f8ad-0eea-44b3-aeee-650f569bd903 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:33.267778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:33.268393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:33.268635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:33.268904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:33.269672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:33.269974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:33.271415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:33.279813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:33.289605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 640/1268] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 10 22:56:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:56:33.295503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:33.392926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515e123-f34c-48b1-b425-eeb4b83105e4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:33.425227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:33.426376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:33.426679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:33.426949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:33.437391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:33.438872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:33.452504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:33.453671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:33.454837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:33.454837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:33.454837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:33.454837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 629/1269] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:33.459442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:33.561721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe05e9-0396-4e0d-a3a2-08741c65b7b1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:33.596888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:33.597643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:33.597926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:33.598189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:33.598847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:33.599141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:33.600516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:33.611505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 641/1270] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:33.617719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:33.649717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:33.715477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0d27-16f1-4345-abab-993e9cd708c9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:33.747599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:33.748204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:33.748479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:33.748743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:33.755747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:33.757172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:56:33.768794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:33.770170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:33.771396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 630/1271] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:56:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:33.776019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:33.834792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:33.878850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6707b1-2742-4d82-b719-a5438e8bbf94 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:33.912046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:33.912583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:33.912860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:33.913133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:33.913815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:33.914118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:33.915576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788ccb0-b116-4697-95dc-89f0ba05590f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:33.928636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 642/1272] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:33.934889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:34.024369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:34.032903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c9dd-a1f0-4b0c-8491-83a04fe9cd9f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:34.065796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cde18-a197-4f59-ab79-6fed9f075cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:34.074114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cde18-a197-4f59-ab79-6fed9f075cac None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:34.074114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cde18-a197-4f59-ab79-6fed9f075cac None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:34.074114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cde18-a197-4f59-ab79-6fed9f075cac None admin] PATH_INFO: `/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/2604f58c168e432d812608a2351610bd/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:34.074114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cde18-a197-4f59-ab79-6fed9f075cac None admin] RBAC: Authorizing `identity:create_grant(project_id=7defdf2789ed4b3b85bc47aa4c32f317, user_id=2604f58c168e432d812608a2351610bd, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:34.074114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cde18-a197-4f59-ab79-6fed9f075cac None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:34.203479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 631/1273] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 10 22:56:33 2022] PUT /identity/v3/projects/7defdf2789ed4b3b85bc47aa4c32f317/users/2604f58c168e432d812608a2351610bd/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:36.314979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222df20f-4234-4031-8b34-874d1a56eca1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:36.315585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222df20f-4234-4031-8b34-874d1a56eca1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:36.315585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222df20f-4234-4031-8b34-874d1a56eca1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:36.316894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 643/1274] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:36.322150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:36.322543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:36.322719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:36.340542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:36.341725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:36.406936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 632/1275] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:36.432059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222df20f-4234-4031-8b34-874d1a56eca1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:36.535155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222df20f-4234-4031-8b34-874d1a56eca1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:36.552952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:36.570011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:36.570565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:36.570872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:36.571147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:36.581080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:56:36.585778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:36.586969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:36.587901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 644/1276] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 10 22:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:36.593120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:36.671630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:36.689461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8b707-5e53-4fcb-99fb-86604b839775 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:36.722034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:36.722625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:36.722929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:36.723194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:36.723842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:36.724138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:36.725476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:36.733138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 633/1277] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 10 22:56:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:36.739415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:36.841017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfdfb7-c736-4a8a-bb12-2d7e3bacd303 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:36.874636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:36.875273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:36.875549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:36.875815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:36.886070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:36.887508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:36.901717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:36.915846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:36.917006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:36.917006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:36.917006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:36.917006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 645/1278] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:36.917269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:37.006680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9c2db-967c-433d-9d77-db826d6bd2d4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:37.039450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:37.040078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:37.040329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:37.040593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:37.041209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:37.042413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:37.042945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:37.054880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 634/1279] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:37.060760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:37.091815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:37.159225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87186f50-542d-4a16-854f-087001c23b94 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:37.192090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:37.192703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:37.192974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:37.193234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:37.200454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:37.201868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:56:37.212621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:37.213797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:37.214900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 646/1280] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:37.219847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:37.224055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:37.315242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2430f1f-78a6-46ed-904b-ccbdaf162820 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:37.348631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:37.349303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:37.349496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:37.349952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:37.350894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:37.351993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:37.353416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:37.370138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 635/1281] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 10 22:56:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:37.376168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:37.465769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:37.475561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8bcb5-f85b-405f-a11d-350eb369c167 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:37.509686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:37.510281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:37.510550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:37.510842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:37.577451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:37.578893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:37.665686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:37.675511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 647/1282] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 10 22:56:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:56:37.681965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:37.744444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:37.780496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5d36-947b-47bc-87ba-912816114d24 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:37.823091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:37.823795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:37.824121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:37.824438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:37.835192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:37.838314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:56:37.851629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:37.852805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:56:37.853954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:56:37.853954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:56:37.853954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:37.853954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 636/1283] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 22:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:37.858435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:37.960514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be5418-e8fe-4b41-a838-dd59ca5583b9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:37.994566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:37.995177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:37.995455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:37.995725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:37.996357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:37.996654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:37.997997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:38.009068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 648/1284] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 10 22:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:38.014775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:38.114342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2805c3-0174-4575-9830-73e4309d355e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:38.127992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:38.149786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:38.150402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:38.150695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:38.150966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:38.158045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:38.159498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:56:38.170752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:38.172081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 10 22:56:38.173007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 637/1285] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 10 22:56:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:38.178389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:38.228691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:38.276915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a0f61-7645-4277-be29-8e4ba7294fcd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:38.309596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:38.310197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:38.310505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:38.310809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:38.311516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:38.311849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:38.313272 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bd074-cccc-495c-a978-dabeed5a2051 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:38.326833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 649/1286] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 10 22:56:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:38.332949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:38.431198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e1005-6a5c-4da2-8574-1ca3e37c67e2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:38.463997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:38.464589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:38.464861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:38.465124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:38.465833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] RBAC: Adding query filter params (role.id=be98a6e301f144d88c992f65bca561fb, scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317, user.id=2604f58c168e432d812608a2351610bd) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:38.466133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:38.467568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fec50-14e0-4271-8caa-f51c6c44a9dd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:38.473841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 638/1287] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 10 22:56:38 2022] GET /identity/v3/role_assignments?user.id=2604f58c168e432d812608a2351610bd&scope.project.id=7defdf2789ed4b3b85bc47aa4c32f317&role.id=be98a6e301f144d88c992f65bca561fb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:40.515798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2279a-2a59-4993-8ae4-b636d91f0770 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:40.516326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2279a-2a59-4993-8ae4-b636d91f0770 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:40.516326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2279a-2a59-4993-8ae4-b636d91f0770 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:40.517661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 650/1288] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:40.522755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:40.523160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:40.523311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:40.541273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:40.542455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:40.572345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:40.608142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 639/1289] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:40.631277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2279a-2a59-4993-8ae4-b636d91f0770 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:40.732191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2279a-2a59-4993-8ae4-b636d91f0770 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:40.743413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:40.767321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:40.767939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:40.768213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:40.768476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:40.769165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:40.770577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:40.779096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:40.780424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:40.781593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 651/1290] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:56:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:40.787071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:40.885775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca182689-3b2a-4aa3-923c-2011a6373471 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:40.919078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:40.919674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:40.919951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:40.920221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:40.920838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:40.921135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:40.922487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2636a0a-7d33-4337-be5e-bcc8a17d722e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:40.927719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 640/1291] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 10 22:56:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:40.933790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:41.034622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a26bbd-f306-4c6f-b7fb-ff634c07214d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:41.068202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:41.068791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:41.069065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:41.069332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:41.069950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:41.070253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:41.071652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de75679-0f09-47fb-a830-bc89655bd3c0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:41.075924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 652/1292] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 10 22:56:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:43.093819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7be92-cde9-437c-acca-adb957a371e6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:43.094352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7be92-cde9-437c-acca-adb957a371e6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:43.094352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7be92-cde9-437c-acca-adb957a371e6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:43.095723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 641/1293] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:56:43.101408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb24b7f-53c6-4b3e-9148-f20fda544b9a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:43.101803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb24b7f-53c6-4b3e-9148-f20fda544b9a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:43.101961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb24b7f-53c6-4b3e-9148-f20fda544b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:43.120476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5cb24b7f-53c6-4b3e-9148-f20fda544b9a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:43.121662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5cb24b7f-53c6-4b3e-9148-f20fda544b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:43.150689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:43.187353 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 653/1294] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:43.211224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7be92-cde9-437c-acca-adb957a371e6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:43.242703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:43.311333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7be92-cde9-437c-acca-adb957a371e6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:43.345464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8930e-77ca-4670-9284-6582c0772c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:43.345999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8930e-77ca-4670-9284-6582c0772c43 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:43.346272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8930e-77ca-4670-9284-6582c0772c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:43.346538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8930e-77ca-4670-9284-6582c0772c43 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:43.347205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8930e-77ca-4670-9284-6582c0772c43 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:43.349140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8930e-77ca-4670-9284-6582c0772c43 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:43.652024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 642/1295] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 10 22:56:43 2022] POST /identity/v3/services => generated 246 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:56:45.680966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4322-1abd-4b35-b99f-532c70954164 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:45.681526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4322-1abd-4b35-b99f-532c70954164 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:45.681526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d4322-1abd-4b35-b99f-532c70954164 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:45.682909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 654/1296] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:45.687918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:45.688367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:45.688468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:45.706676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:45.707865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:45.780217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 643/1297] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:45.803134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d4322-1abd-4b35-b99f-532c70954164 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:45.906109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d4322-1abd-4b35-b99f-532c70954164 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:45.940233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:45.940790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:45.941069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:45.941368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:45.941940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:45.943317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:45.951747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:45.953821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:45.956060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 655/1298] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:56:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:45.961104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:45.979315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:46.062397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a742b-82a5-41d4-b9a9-b395471fb0fd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:46.094679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:46.095280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:46.095558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:46.095829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:46.096508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:46.096745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:46.098618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:46.103562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 644/1299] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 10 22:56:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:46.109598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:46.141843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:46.208063 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f268e-ba7d-48ae-99f5-6744f14998f0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:46.241903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:46.242503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:46.242915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:46.242915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:46.244046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] RBAC: Adding query filter params (interface=public, service_id=d4c9c1705e0b4457b885d074323fb8be, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:46.244306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:46.245660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e814db-3d02-45c0-9ec8-9fe4d9afccfe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:46.250580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 656/1300] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 10 22:56:46 2022] GET /identity/v3/endpoints?service_id=d4c9c1705e0b4457b885d074323fb8be&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:46.256371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:46.338963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:46.356794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af456c-7a07-478d-a45f-9c67f5ee0550 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:46.389092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82824f2-f4b1-4158-b243-07e62a7d346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:46.389681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82824f2-f4b1-4158-b243-07e62a7d346e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:46.389958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82824f2-f4b1-4158-b243-07e62a7d346e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:46.390222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82824f2-f4b1-4158-b243-07e62a7d346e None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:46.390842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82824f2-f4b1-4158-b243-07e62a7d346e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:46.392201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82824f2-f4b1-4158-b243-07e62a7d346e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:46.399390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 645/1301] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 10 22:56:46 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:56:48.388006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c2572-bcd5-418a-9acb-5bedff4b7c80 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:48.388561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c2572-bcd5-418a-9acb-5bedff4b7c80 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:48.388561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c2572-bcd5-418a-9acb-5bedff4b7c80 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:48.389892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 657/1302] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 22:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:48.395059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:48.395524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:48.395626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:48.413640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:48.414913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:48.485506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 646/1303] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 22:56:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:56:48.509542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c2572-bcd5-418a-9acb-5bedff4b7c80 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:48.622442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:48.629755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c2572-bcd5-418a-9acb-5bedff4b7c80 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:48.662801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:48.663348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:48.663619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:48.663883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:48.664446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:48.665806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:56:48.674116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:56:48.675387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 10 22:56:48.677100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 658/1304] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 10 22:56:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:56:48.681842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:48.758018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:48.779514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87f18c-0142-43e4-8f67-ada49c4694ee None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:48.813798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:48.814417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:48.814708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:48.814976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:48.815595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:48.815888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:48.817835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaeb72-d771-46fa-8682-b281f4be6583 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:48.823665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 647/1305] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 10 22:56:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:48.829583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:48.946246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1c4b3-2265-42a3-ab86-0fe0ac8b6190 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:48.987402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be58263-7975-47b6-9de7-be850d1cf109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:48.987402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58263-7975-47b6-9de7-be850d1cf109 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:48.987402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58263-7975-47b6-9de7-be850d1cf109 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:48.987402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be58263-7975-47b6-9de7-be850d1cf109 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:48.987924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be58263-7975-47b6-9de7-be850d1cf109 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:48.987924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be58263-7975-47b6-9de7-be850d1cf109 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:49.125257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 659/1306] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 22:56:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:56:58.851369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=30 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:59.120184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:05.042639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967bc79-fed0-442d-ba68-0072d6bf392b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:05.043215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967bc79-fed0-442d-ba68-0072d6bf392b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:05.043215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967bc79-fed0-442d-ba68-0072d6bf392b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:05.050345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb498ed-02f5-402b-884b-1f3b146de391 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:05.050857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb498ed-02f5-402b-884b-1f3b146de391 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:05.050950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb498ed-02f5-402b-884b-1f3b146de391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:05.053776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 648/1307] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:05.137534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1cb498ed-02f5-402b-884b-1f3b146de391 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:05.138815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1cb498ed-02f5-402b-884b-1f3b146de391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:05.267106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 660/1308] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:05:05.333815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca4fa1-ab9c-491c-bb59-a16255749bb5 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:05.334105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca4fa1-ab9c-491c-bb59-a16255749bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:05.334381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca4fa1-ab9c-491c-bb59-a16255749bb5 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:05.336058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 649/1309] 10.222.0.78 () {54 vars in 965 bytes} [Wed Aug 10 23:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:05.346535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace674c9-e415-42cb-ae19-4f8878380704 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:05.346774 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace674c9-e415-42cb-ae19-4f8878380704 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:05.347074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace674c9-e415-42cb-ae19-4f8878380704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:05.508682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ace674c9-e415-42cb-ae19-4f8878380704 None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:05.510000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ace674c9-e415-42cb-ae19-4f8878380704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:05.564518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:05.645484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 661/1310] 10.222.0.78 () {58 vars in 1062 bytes} [Wed Aug 10 23:05:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:05.654676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca4fa1-ab9c-491c-bb59-a16255749bb5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:05.656664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0eca4fa1-ab9c-491c-bb59-a16255749bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:05.797762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca4fa1-ab9c-491c-bb59-a16255749bb5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:05.901177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:05.901828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:05.902245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:05.902515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:05.904553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:05.977947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:06.046202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:06.049205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2daf1-d8df-4e45-a72d-0a93b6d36016 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:06.071157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 650/1311] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:05:08.382432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674257c-d7e4-4a2f-b713-844d96554077 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:08.383013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674257c-d7e4-4a2f-b713-844d96554077 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:08.383013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674257c-d7e4-4a2f-b713-844d96554077 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:08.384360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 662/1312] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:08.390248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7f1ba-989d-4a5e-b8d1-7b181110b55c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:08.390796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7f1ba-989d-4a5e-b8d1-7b181110b55c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:08.390983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7f1ba-989d-4a5e-b8d1-7b181110b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:08.471654 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-fcf7f1ba-989d-4a5e-b8d1-7b181110b55c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:08.472882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fcf7f1ba-989d-4a5e-b8d1-7b181110b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:08.579798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 651/1313] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:08.619271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674257c-d7e4-4a2f-b713-844d96554077 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:08.620931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8674257c-d7e4-4a2f-b713-844d96554077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:08.724182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674257c-d7e4-4a2f-b713-844d96554077 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:08.798270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:08.801824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:08.802713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:08.803125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:08.803523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:08.806085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:08.948053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:08.949398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4458-0fff-497c-8483-69d285446310 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:08.968043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 663/1314] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:11.515631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b24ed-30ea-4d85-b5da-50535a832c54 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:11.516151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b24ed-30ea-4d85-b5da-50535a832c54 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:11.516272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b24ed-30ea-4d85-b5da-50535a832c54 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:11.517614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 652/1315] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:11.523083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd2101-1517-4bc8-86bf-30e21303539a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:11.523530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd2101-1517-4bc8-86bf-30e21303539a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:11.523636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd2101-1517-4bc8-86bf-30e21303539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:11.592123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-63bd2101-1517-4bc8-86bf-30e21303539a None None] MFA Rules not processed for user `5d64a6dd6a5e42d0906f4f83111df269`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:11.593337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-63bd2101-1517-4bc8-86bf-30e21303539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:11.644588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:11.688442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 664/1316] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:11.728076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b24ed-30ea-4d85-b5da-50535a832c54 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:11.729611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-446b24ed-30ea-4d85-b5da-50535a832c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:11.739934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:11.831668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b24ed-30ea-4d85-b5da-50535a832c54 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:11.902910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:11.909031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:11.909031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:11.909031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:11.909031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:11.968172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:12.013771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:12.015251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41b447-329c-402b-972c-0139a2d30c82 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:12.035799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 653/1317] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:14.822754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6d6d5-5563-4ec0-a418-b46d49691b0f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:14.823302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6d6d5-5563-4ec0-a418-b46d49691b0f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:14.823302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6d6d5-5563-4ec0-a418-b46d49691b0f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:14.824657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 665/1318] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:14.829869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfbcd2-a745-4fef-8fba-8d2ffe9972e3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:14.830260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfbcd2-a745-4fef-8fba-8d2ffe9972e3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:14.830408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfbcd2-a745-4fef-8fba-8d2ffe9972e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:14.894032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c5bfbcd2-a745-4fef-8fba-8d2ffe9972e3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:14.895250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c5bfbcd2-a745-4fef-8fba-8d2ffe9972e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:14.989189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 654/1319] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:15.028158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6d6d5-5563-4ec0-a418-b46d49691b0f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:15.029852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-05f6d6d5-5563-4ec0-a418-b46d49691b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:15.131462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6d6d5-5563-4ec0-a418-b46d49691b0f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:15.211653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:15.212242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:15.212518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:15.212786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:15.214370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:15.222249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:15.321737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:15.323111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455f23e-52bd-4635-b27e-5a90ef50d62a service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:15.341494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 666/1320] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:18.383852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93fff1-5a42-4dd1-b707-71f6be514676 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:18.384453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93fff1-5a42-4dd1-b707-71f6be514676 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:18.384453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93fff1-5a42-4dd1-b707-71f6be514676 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:18.385715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 655/1321] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:18.391229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023628f-87e7-4722-a38c-83c59d747b05 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:18.391689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023628f-87e7-4722-a38c-83c59d747b05 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:18.391800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023628f-87e7-4722-a38c-83c59d747b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:18.459920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7023628f-87e7-4722-a38c-83c59d747b05 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:18.461392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7023628f-87e7-4722-a38c-83c59d747b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:18.518561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:18.558025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 667/1322] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:18.608791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09fce5-5eda-4190-aec9-05da97df78bb None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:18.609208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09fce5-5eda-4190-aec9-05da97df78bb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:18.609545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09fce5-5eda-4190-aec9-05da97df78bb None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:18.611597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 656/1323] 10.222.0.78 () {54 vars in 965 bytes} [Wed Aug 10 23:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:18.617803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2329a13-82f2-4442-8d67-f12f13c72979 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:18.618249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2329a13-82f2-4442-8d67-f12f13c72979 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:18.618534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2329a13-82f2-4442-8d67-f12f13c72979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:18.703157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f2329a13-82f2-4442-8d67-f12f13c72979 None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:18.705171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2329a13-82f2-4442-8d67-f12f13c72979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:18.786860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 668/1324] 10.222.0.78 () {58 vars in 1062 bytes} [Wed Aug 10 23:05:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:18.796123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09fce5-5eda-4190-aec9-05da97df78bb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:18.797628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cc09fce5-5eda-4190-aec9-05da97df78bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:18.819156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:18.903168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09fce5-5eda-4190-aec9-05da97df78bb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:18.976902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:18.977425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:18.977702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:18.977971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:18.979670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:19.057269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:19.095588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:19.097180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556988aa-e02a-4455-b8cc-b4fe456f5c06 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:19.116746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 657/1325] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:05:21.405124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b9637-8e87-4a9b-a004-12649eea5126 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:21.405701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b9637-8e87-4a9b-a004-12649eea5126 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:21.405701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b9637-8e87-4a9b-a004-12649eea5126 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:21.407543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 669/1326] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:21.412963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4094-22e8-43d9-ba4a-1fda65636178 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:21.413376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4094-22e8-43d9-ba4a-1fda65636178 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:21.413519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4094-22e8-43d9-ba4a-1fda65636178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:21.483497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-06bf4094-22e8-43d9-ba4a-1fda65636178 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:21.485515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-06bf4094-22e8-43d9-ba4a-1fda65636178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:21.585830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 658/1327] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:21.623387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b9637-8e87-4a9b-a004-12649eea5126 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:21.624982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-455b9637-8e87-4a9b-a004-12649eea5126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:21.685268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:21.734217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b9637-8e87-4a9b-a004-12649eea5126 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:21.808966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:21.809574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:21.809852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:21.810126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:21.811757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:21.918803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:21.921306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:21.922911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e313d-5a87-4a37-9009-870046a8f38e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:21.940817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 670/1328] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:24.025045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d687e-cfa8-4056-bcc7-12bfd5973151 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:24.025588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d687e-cfa8-4056-bcc7-12bfd5973151 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:24.025588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d687e-cfa8-4056-bcc7-12bfd5973151 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:24.026965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 659/1329] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:24.032776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c8dc-fe38-418c-84a6-20c2205b400f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:24.033194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c8dc-fe38-418c-84a6-20c2205b400f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:24.033323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c8dc-fe38-418c-84a6-20c2205b400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:24.099712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1686c8dc-fe38-418c-84a6-20c2205b400f None None] MFA Rules not processed for user `5d64a6dd6a5e42d0906f4f83111df269`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:24.100921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1686c8dc-fe38-418c-84a6-20c2205b400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:24.199579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 671/1330] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:24.238430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d687e-cfa8-4056-bcc7-12bfd5973151 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:24.239989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9a8d687e-cfa8-4056-bcc7-12bfd5973151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:24.270871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:24.371834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d687e-cfa8-4056-bcc7-12bfd5973151 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:24.453762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:24.454368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:24.454645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:24.454987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:24.456521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:24.534958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:24.565683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:24.567605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37983646-cf23-4148-84c3-3485d4346c2d service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:24.586072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 660/1331] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:27.382821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f22b-494a-4870-a7da-17c6820b93e1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:27.383386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f22b-494a-4870-a7da-17c6820b93e1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:27.383386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f22b-494a-4870-a7da-17c6820b93e1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:27.384702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 672/1332] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:27.389855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64f789-b005-4f85-816e-a6092cadb9a0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:27.390235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64f789-b005-4f85-816e-a6092cadb9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:27.390406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64f789-b005-4f85-816e-a6092cadb9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:27.456908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4b64f789-b005-4f85-816e-a6092cadb9a0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:27.458150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4b64f789-b005-4f85-816e-a6092cadb9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:27.551844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 661/1333] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:27.588366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f22b-494a-4870-a7da-17c6820b93e1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:27.589907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0317f22b-494a-4870-a7da-17c6820b93e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:27.657538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:27.691254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f22b-494a-4870-a7da-17c6820b93e1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:27.764335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:27.764932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:27.765209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:27.767235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:27.767235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:27.882045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:27.883726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202154-ce4c-433a-9b90-721f1daa074d service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:27.894295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:27.902483 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 673/1334] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:29.958967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9e66-195d-4744-937e-5c5245ae7327 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:29.959542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9e66-195d-4744-937e-5c5245ae7327 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:29.959784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9e66-195d-4744-937e-5c5245ae7327 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:29.961186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 662/1335] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:29.966359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65903-22f6-436c-b043-516ce3375dcb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:29.966813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65903-22f6-436c-b043-516ce3375dcb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:29.966921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65903-22f6-436c-b043-516ce3375dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:30.031901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3cd65903-22f6-436c-b043-516ce3375dcb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:30.033110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3cd65903-22f6-436c-b043-516ce3375dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:30.129102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 674/1336] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:30.168296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9e66-195d-4744-937e-5c5245ae7327 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:30.169804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-272e9e66-195d-4744-937e-5c5245ae7327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:30.209759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:30.271248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9e66-195d-4744-937e-5c5245ae7327 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:30.344016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:30.344619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:30.344899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:30.345172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:30.346803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:30.435134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:30.460140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:30.461521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1d50b-0007-4846-b5d9-1c8d030e7c21 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:30.479489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 663/1337] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:33.241614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914784d4-4594-40b2-9e9f-9582385b013a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:33.242187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914784d4-4594-40b2-9e9f-9582385b013a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:33.242187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914784d4-4594-40b2-9e9f-9582385b013a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:33.243540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 675/1338] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:33.248703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615d0c0-c737-4223-bc48-7d06844c16f4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:33.249147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615d0c0-c737-4223-bc48-7d06844c16f4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:33.249244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615d0c0-c737-4223-bc48-7d06844c16f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:33.315086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6615d0c0-c737-4223-bc48-7d06844c16f4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:33.316304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6615d0c0-c737-4223-bc48-7d06844c16f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:33.411090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 664/1339] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:33.447699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914784d4-4594-40b2-9e9f-9582385b013a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:33.449236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-914784d4-4594-40b2-9e9f-9582385b013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:33.525575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:33.553528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914784d4-4594-40b2-9e9f-9582385b013a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:33.625966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:33.626562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:33.626867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:33.627132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:33.628705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:33.733418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:33.735647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b2b68-4fb2-4603-86f3-9245e43dd5c6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:33.753726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 676/1340] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:38.151525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104655d8-c873-4676-9021-65486848bf90 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:38.151525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104655d8-c873-4676-9021-65486848bf90 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:38.151982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104655d8-c873-4676-9021-65486848bf90 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:38.153038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 665/1341] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:38.159403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe2998-4b4d-4983-a75f-f1d10009c811 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:38.159875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe2998-4b4d-4983-a75f-f1d10009c811 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:38.159998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe2998-4b4d-4983-a75f-f1d10009c811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:38.165508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:38.228254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d7fe2998-4b4d-4983-a75f-f1d10009c811 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:38.229444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d7fe2998-4b4d-4983-a75f-f1d10009c811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:38.326827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 677/1342] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:38.365777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104655d8-c873-4676-9021-65486848bf90 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:38.367340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-104655d8-c873-4676-9021-65486848bf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:38.418364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:38.476052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104655d8-c873-4676-9021-65486848bf90 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:38.550628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:38.551257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:38.551528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:38.551793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:38.553351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:38.644208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:38.661574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:38.663233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c2e1f-339a-4621-963d-a8cc9c1e0086 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:38.682673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 666/1343] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:41.917352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e3a0-4091-4425-96eb-a44c0d8f4fee None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:41.917900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e3a0-4091-4425-96eb-a44c0d8f4fee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:41.917900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e3a0-4091-4425-96eb-a44c0d8f4fee None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:41.919455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 678/1344] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:41.924737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae546b-d279-426c-a171-26f13d9b28b1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:41.925161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae546b-d279-426c-a171-26f13d9b28b1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:41.925415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae546b-d279-426c-a171-26f13d9b28b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:41.993242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7fae546b-d279-426c-a171-26f13d9b28b1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:41.994459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7fae546b-d279-426c-a171-26f13d9b28b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:42.092626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 667/1345] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:42.131381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489e3a0-4091-4425-96eb-a44c0d8f4fee None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:42.132901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1489e3a0-4091-4425-96eb-a44c0d8f4fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:42.220164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:42.239841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489e3a0-4091-4425-96eb-a44c0d8f4fee None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:42.318191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:42.318949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:42.319227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:42.319495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:42.321204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:42.438437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:42.439913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8755d-2d35-4f2c-91e9-dd1078330690 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:42.468821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 679/1346] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:46.989731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a73f-fb3b-4b93-b46c-2efaab88cf46 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:46.990284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a73f-fb3b-4b93-b46c-2efaab88cf46 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:46.990284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a73f-fb3b-4b93-b46c-2efaab88cf46 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:46.991735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 668/1347] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:46.997754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec12a-145c-45ec-be10-77441c9de848 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:46.998222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec12a-145c-45ec-be10-77441c9de848 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:46.998326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec12a-145c-45ec-be10-77441c9de848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:47.016054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:47.071205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7deec12a-145c-45ec-be10-77441c9de848 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:47.072428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7deec12a-145c-45ec-be10-77441c9de848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:47.174720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 680/1348] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:47.213614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a73f-fb3b-4b93-b46c-2efaab88cf46 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:47.215668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8e99a73f-fb3b-4b93-b46c-2efaab88cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:47.277882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:47.327157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a73f-fb3b-4b93-b46c-2efaab88cf46 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:47.401115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:47.401736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:47.402425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:47.402556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:47.404303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:47.517306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:47.520858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:47.522304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b549e-25ac-4a10-a51a-dd8a41c401ea service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:47.543899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 669/1349] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:52.572187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a700d84-28b2-462c-acaa-7f826e438ec6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:52.572703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a700d84-28b2-462c-acaa-7f826e438ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:52.572703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a700d84-28b2-462c-acaa-7f826e438ec6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:52.574942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 681/1350] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:05:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:52.581189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed929fc-5b7d-4312-89fc-9e0eec282038 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:52.581637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed929fc-5b7d-4312-89fc-9e0eec282038 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:52.581746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed929fc-5b7d-4312-89fc-9e0eec282038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:52.703679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-eed929fc-5b7d-4312-89fc-9e0eec282038 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:52.706644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-eed929fc-5b7d-4312-89fc-9e0eec282038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:52.846803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 670/1351] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:05:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:57.019497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725fa8-c2d0-46b6-8fd8-82d521e4b865 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:57.020004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725fa8-c2d0-46b6-8fd8-82d521e4b865 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:57.020004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725fa8-c2d0-46b6-8fd8-82d521e4b865 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:57.021496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 682/1352] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:57.026394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf9553-9413-4b2c-ac2c-6f7fc1248562 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:57.026895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf9553-9413-4b2c-ac2c-6f7fc1248562 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:57.027102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf9553-9413-4b2c-ac2c-6f7fc1248562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:57.090070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:57.114546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-28bf9553-9413-4b2c-ac2c-6f7fc1248562 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:57.115833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-28bf9553-9413-4b2c-ac2c-6f7fc1248562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:57.172652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=59 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:57.227664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 671/1353] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:05:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:05:58.644998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72465e0-bb4e-41b1-b058-5fad0d755767 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:58.645663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72465e0-bb4e-41b1-b058-5fad0d755767 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:58.645663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72465e0-bb4e-41b1-b058-5fad0d755767 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:58.646921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 683/1354] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:05:58.651954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea667c-a6b4-47d2-9683-c081b7f446c4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:58.656195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea667c-a6b4-47d2-9683-c081b7f446c4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:58.656195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea667c-a6b4-47d2-9683-c081b7f446c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:58.729006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1bea667c-a6b4-47d2-9683-c081b7f446c4 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:05:58.730223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1bea667c-a6b4-47d2-9683-c081b7f446c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:58.777947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:58.845802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 672/1355] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:05:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:02.296459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05a37c-05b1-433f-b87a-b7512d246322 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:02.297030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05a37c-05b1-433f-b87a-b7512d246322 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:02.297030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05a37c-05b1-433f-b87a-b7512d246322 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:02.298414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 684/1356] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:02.304025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a94ba-b6bf-4775-9474-eace606e94a9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:02.304470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a94ba-b6bf-4775-9474-eace606e94a9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:02.304586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a94ba-b6bf-4775-9474-eace606e94a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:02.393227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ec7a94ba-b6bf-4775-9474-eace606e94a9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:02.394846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ec7a94ba-b6bf-4775-9474-eace606e94a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:02.516613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 673/1357] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:02.572441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec1ecb-7288-43fb-b94b-b29fe1e42d8b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:02.572920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec1ecb-7288-43fb-b94b-b29fe1e42d8b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:02.573366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec1ecb-7288-43fb-b94b-b29fe1e42d8b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:02.574723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 685/1358] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 10 23:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:02.584940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db905b8c-6bf7-4be3-9d14-fff2dddaf694 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:02.585449 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db905b8c-6bf7-4be3-9d14-fff2dddaf694 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:02.586001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db905b8c-6bf7-4be3-9d14-fff2dddaf694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:02.604393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:02.684981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-db905b8c-6bf7-4be3-9d14-fff2dddaf694 None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:02.686268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-db905b8c-6bf7-4be3-9d14-fff2dddaf694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:02.826951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 674/1359] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 10 23:06:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:02.836052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec1ecb-7288-43fb-b94b-b29fe1e42d8b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:02.837595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-40ec1ecb-7288-43fb-b94b-b29fe1e42d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:03.006370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec1ecb-7288-43fb-b94b-b29fe1e42d8b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:03.095877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:03.142584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:03.143370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:03.143814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:03.144178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:03.145906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:03.241147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:03.242916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b60ac-b096-413e-99a6-ccc646e46570 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:03.261636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 686/1360] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:06:05.503205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a825-f54e-46c8-b79b-146e13e82e80 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:05.503753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a825-f54e-46c8-b79b-146e13e82e80 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:05.503753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a825-f54e-46c8-b79b-146e13e82e80 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:05.505223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 675/1361] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:05.510266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f9bb-628d-40b2-bf02-b65621c78a34 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:05.510782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f9bb-628d-40b2-bf02-b65621c78a34 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:05.511092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f9bb-628d-40b2-bf02-b65621c78a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:05.599621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4fd5f9bb-628d-40b2-bf02-b65621c78a34 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:05.600957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4fd5f9bb-628d-40b2-bf02-b65621c78a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:05.642543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:05.703756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 687/1362] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:05.749757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967a825-f54e-46c8-b79b-146e13e82e80 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:05.751288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0967a825-f54e-46c8-b79b-146e13e82e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:05.827148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:05.934562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967a825-f54e-46c8-b79b-146e13e82e80 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:06.079763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:06.080452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:06.080740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:06.081015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:06.082598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:06.107617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:06.177515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:06.179153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b5ef6-0728-4416-9bd4-5785893bb90e service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:06.198063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 676/1363] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:08.683021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2337a-48f6-4e15-8fd1-f28802cd2508 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:08.683549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2337a-48f6-4e15-8fd1-f28802cd2508 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:08.683549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2337a-48f6-4e15-8fd1-f28802cd2508 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:08.685095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 688/1364] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:08.690806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0e46e-48af-4fd0-a56e-f4755f959e34 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:08.691320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0e46e-48af-4fd0-a56e-f4755f959e34 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:08.691432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0e46e-48af-4fd0-a56e-f4755f959e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:08.775140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-24f0e46e-48af-4fd0-a56e-f4755f959e34 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:08.776509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-24f0e46e-48af-4fd0-a56e-f4755f959e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:08.887836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:08.897268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 677/1365] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:08.937967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ae30a-378e-47f6-9a84-76751a7f6dfc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:08.938206 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ae30a-378e-47f6-9a84-76751a7f6dfc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:08.938512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ae30a-378e-47f6-9a84-76751a7f6dfc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:08.940455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 689/1366] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 10 23:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:08.947471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748abd9-aa70-4400-9400-963026938dc8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:08.947752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748abd9-aa70-4400-9400-963026938dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:08.948020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748abd9-aa70-4400-9400-963026938dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:09.030373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7748abd9-aa70-4400-9400-963026938dc8 None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:09.031593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7748abd9-aa70-4400-9400-963026938dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:09.155989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 678/1367] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 10 23:06:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:09.163649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ae30a-378e-47f6-9a84-76751a7f6dfc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:09.165139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-078ae30a-378e-47f6-9a84-76751a7f6dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:09.343495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ae30a-378e-47f6-9a84-76751a7f6dfc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:09.364849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:09.473778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:09.474889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:09.475400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:09.475883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:09.477716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:09.580992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:09.583102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afee86b-6c6e-4262-8cc6-60e2c585a375 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:09.604010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 690/1368] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:10.568487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd42dd-89ff-4e27-9462-39019caa446d None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:10.569008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd42dd-89ff-4e27-9462-39019caa446d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:10.569008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd42dd-89ff-4e27-9462-39019caa446d None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:10.570366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 679/1369] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:10.575544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ee310-9571-4d23-bd79-77bc06ab0023 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:10.575977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ee310-9571-4d23-bd79-77bc06ab0023 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:10.576669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ee310-9571-4d23-bd79-77bc06ab0023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:10.616703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:10.653439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e64ee310-9571-4d23-bd79-77bc06ab0023 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:10.654909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e64ee310-9571-4d23-bd79-77bc06ab0023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:10.768759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 691/1370] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:11.147978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e806-e4e3-495f-ab37-aef8f01a8ddc None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:11.148374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e806-e4e3-495f-ab37-aef8f01a8ddc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:11.148755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074e806-e4e3-495f-ab37-aef8f01a8ddc None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:11.150216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 680/1371] 10.222.0.78 () {54 vars in 964 bytes} [Wed Aug 10 23:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:11.156000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe62c5-8477-46be-90bf-8aaf4bf1edf6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:11.156401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe62c5-8477-46be-90bf-8aaf4bf1edf6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:11.156783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe62c5-8477-46be-90bf-8aaf4bf1edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:11.246511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-eebe62c5-8477-46be-90bf-8aaf4bf1edf6 None None] MFA Rules not processed for user `54fff64dfcf8434d867c028a72c17a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:11.247776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eebe62c5-8477-46be-90bf-8aaf4bf1edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:11.303305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:11.335629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 692/1372] 10.222.0.78 () {58 vars in 1061 bytes} [Wed Aug 10 23:06:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:06:11.343396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074e806-e4e3-495f-ab37-aef8f01a8ddc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:11.344892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4074e806-e4e3-495f-ab37-aef8f01a8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:11.397751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:11.465111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074e806-e4e3-495f-ab37-aef8f01a8ddc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:11.543626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:11.544221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:11.544494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:11.545275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:11.546876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:11.650084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:11.665155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:11.666830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057b096-a21c-458a-9aa7-59585e7ea01a service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:11.685741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 681/1373] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:11.895883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03aded-4ca0-4ad3-a8a3-4f73c1f6e33f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:11.896350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03aded-4ca0-4ad3-a8a3-4f73c1f6e33f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:11.896598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03aded-4ca0-4ad3-a8a3-4f73c1f6e33f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:11.897754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 693/1374] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:11.903728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f4974-df7a-4174-97c7-2a673207ce8c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:11.903886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f4974-df7a-4174-97c7-2a673207ce8c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:11.904755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f4974-df7a-4174-97c7-2a673207ce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:11.973141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4e7f4974-df7a-4174-97c7-2a673207ce8c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:11.974997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e7f4974-df7a-4174-97c7-2a673207ce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:12.097326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 682/1375] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:12.142420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03aded-4ca0-4ad3-a8a3-4f73c1f6e33f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:12.145691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bc03aded-4ca0-4ad3-a8a3-4f73c1f6e33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:12.292899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03aded-4ca0-4ad3-a8a3-4f73c1f6e33f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:12.345607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:12.407871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:12.408389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:12.408660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:12.408926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:12.410547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:12.504652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:12.506252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1529a-a661-4eee-9d06-2f4e302385d4 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:12.525431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 694/1376] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:14.707350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd4dd8-379c-4c19-9444-1b419319b3a8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:14.707890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd4dd8-379c-4c19-9444-1b419319b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:14.707890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd4dd8-379c-4c19-9444-1b419319b3a8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:14.709263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 683/1377] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:14.714706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99448-d6a2-442b-b995-d9eea1b86406 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:14.715123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99448-d6a2-442b-b995-d9eea1b86406 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:14.715245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99448-d6a2-442b-b995-d9eea1b86406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:14.786522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d2f99448-d6a2-442b-b995-d9eea1b86406 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:14.788102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d2f99448-d6a2-442b-b995-d9eea1b86406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:14.804767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:14.905739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 695/1378] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:14.970265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd4dd8-379c-4c19-9444-1b419319b3a8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:14.971738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2bfd4dd8-379c-4c19-9444-1b419319b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:15.036929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:15.121464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd4dd8-379c-4c19-9444-1b419319b3a8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:15.243703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:15.244222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:15.244494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:15.244762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:15.246352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:15.265586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:15.349411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:15.351046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566ba06-60c6-4ca8-9c45-fcf43f75d704 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:15.387531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 684/1379] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:17.564195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ba1f-8d90-488a-9db8-0eff223fc918 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:17.564746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ba1f-8d90-488a-9db8-0eff223fc918 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:17.564746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ba1f-8d90-488a-9db8-0eff223fc918 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:17.567020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 696/1380] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:17.571282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dd071-7d29-4fa0-bc4a-f6c7c2d4526e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:17.571708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dd071-7d29-4fa0-bc4a-f6c7c2d4526e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:17.571909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dd071-7d29-4fa0-bc4a-f6c7c2d4526e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:17.639238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-994dd071-7d29-4fa0-bc4a-f6c7c2d4526e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:17.640414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-994dd071-7d29-4fa0-bc4a-f6c7c2d4526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:17.737535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:17.744971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 685/1381] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:17.790093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8ba1f-8d90-488a-9db8-0eff223fc918 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:17.791742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-abf8ba1f-8d90-488a-9db8-0eff223fc918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:17.941805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:17.952671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8ba1f-8d90-488a-9db8-0eff223fc918 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:18.074882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:18.076174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:18.078050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:18.078432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:18.080342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:18.178390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:18.180244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed2ad1-d008-402d-9467-7aec403dc104 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:18.196069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:18.201012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 697/1382] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:20.333965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb925658-a6df-4f35-99c5-4ef1191f6821 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:20.334515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb925658-a6df-4f35-99c5-4ef1191f6821 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:20.334515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb925658-a6df-4f35-99c5-4ef1191f6821 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:20.336037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 686/1383] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:20.341660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08082-7d09-4ef1-8125-35d15a8cdeff None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:20.342144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08082-7d09-4ef1-8125-35d15a8cdeff None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:20.342258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08082-7d09-4ef1-8125-35d15a8cdeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:20.410769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c9d08082-7d09-4ef1-8125-35d15a8cdeff None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:20.411993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c9d08082-7d09-4ef1-8125-35d15a8cdeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:20.540136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 698/1384] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:20.582823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb925658-a6df-4f35-99c5-4ef1191f6821 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:20.584723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cb925658-a6df-4f35-99c5-4ef1191f6821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:20.758502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb925658-a6df-4f35-99c5-4ef1191f6821 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:20.835955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:20.895148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:20.895867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:20.896202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:20.896533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:20.898519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:21.008478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:21.010313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de262ae-6940-4d81-8a80-3ec38d14aa22 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:21.032965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 687/1385] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:23.486961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86120-5031-4ad0-b32b-92a774e672e7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:23.487552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86120-5031-4ad0-b32b-92a774e672e7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:23.487552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86120-5031-4ad0-b32b-92a774e672e7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:23.488895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 699/1386] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:23.495694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a1a6d-22c8-42f8-b54a-952d3cd3f5f8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:23.496152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a1a6d-22c8-42f8-b54a-952d3cd3f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:23.496265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a1a6d-22c8-42f8-b54a-952d3cd3f5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:23.565906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-653a1a6d-22c8-42f8-b54a-952d3cd3f5f8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:23.567198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-653a1a6d-22c8-42f8-b54a-952d3cd3f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:23.610479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:23.678805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 688/1387] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:23.740356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86120-5031-4ad0-b32b-92a774e672e7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:23.742478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-65e86120-5031-4ad0-b32b-92a774e672e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:23.829329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:23.922337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86120-5031-4ad0-b32b-92a774e672e7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:24.076259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:24.077003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:24.077367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:24.077730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:24.080793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:24.133466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:24.190045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:24.191663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55dc171-0986-40a5-af64-d71fccca1525 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:24.211674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 700/1388] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:26.347085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4b41-c598-441d-9300-7de26590ab21 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:26.347661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4b41-c598-441d-9300-7de26590ab21 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:26.347661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4b41-c598-441d-9300-7de26590ab21 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:26.349323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 689/1389] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:26.354606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97b0fe-c975-46f7-91a9-fa5e7309de4a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:26.355115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97b0fe-c975-46f7-91a9-fa5e7309de4a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:26.355199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97b0fe-c975-46f7-91a9-fa5e7309de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:26.438861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9e97b0fe-c975-46f7-91a9-fa5e7309de4a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:26.440390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e97b0fe-c975-46f7-91a9-fa5e7309de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:26.557486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 701/1390] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:26.603254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc4b41-c598-441d-9300-7de26590ab21 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:26.604657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d8cc4b41-c598-441d-9300-7de26590ab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:26.768645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc4b41-c598-441d-9300-7de26590ab21 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:26.787678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:26.894944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:26.895546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:26.895824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:26.896095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:26.897691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:26.988187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:26.989920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b5b26-7a90-410b-8340-00a75c2e7ee1 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:27.008091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 690/1391] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:27.140141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916736-523b-46c5-841a-7eb3917e11a3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:27.140486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916736-523b-46c5-841a-7eb3917e11a3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:27.140753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916736-523b-46c5-841a-7eb3917e11a3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:27.142095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 702/1392] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:27.147629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b323edf-b8fa-4d31-a37d-a304f11b1d27 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:27.147857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b323edf-b8fa-4d31-a37d-a304f11b1d27 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:27.148137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b323edf-b8fa-4d31-a37d-a304f11b1d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:27.174729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:27.215195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9b323edf-b8fa-4d31-a37d-a304f11b1d27 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:27.216468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9b323edf-b8fa-4d31-a37d-a304f11b1d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:27.313712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 691/1393] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:06:27.371722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1be98-2df8-4a98-a26a-6b1402606990 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:27.372899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1be98-2df8-4a98-a26a-6b1402606990 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:27.372899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1be98-2df8-4a98-a26a-6b1402606990 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:27.374728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 703/1394] 10.222.0.78 () {54 vars in 967 bytes} [Wed Aug 10 23:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:27.380217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1113fd-ebd6-4f66-b903-7842decb16bc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:27.380768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1113fd-ebd6-4f66-b903-7842decb16bc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:27.380768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1113fd-ebd6-4f66-b903-7842decb16bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:27.466390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5c1113fd-ebd6-4f66-b903-7842decb16bc None None] MFA Rules not processed for user `f28dc00f9c744cc4b103b8e01d65f75f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:27.467661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5c1113fd-ebd6-4f66-b903-7842decb16bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:27.511811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:27.549127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 692/1395] 10.222.0.78 () {58 vars in 1064 bytes} [Wed Aug 10 23:06:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:27.557037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1be98-2df8-4a98-a26a-6b1402606990 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:27.558551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1cc1be98-2df8-4a98-a26a-6b1402606990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:27.572676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:27.665398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1be98-2df8-4a98-a26a-6b1402606990 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:27.741226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:27.741750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:27.742027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:27.742296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:27.744027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:27.815803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:27.857690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:27.859422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21a52-8145-473f-9e06-d81e8e5b0b9d service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:27.878490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 704/1396] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:06:28.779353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679628af-b121-4684-8412-9e674f13ef16 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:28.779869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679628af-b121-4684-8412-9e674f13ef16 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:28.780011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679628af-b121-4684-8412-9e674f13ef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:28.847940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-679628af-b121-4684-8412-9e674f13ef16 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:28.849137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-679628af-b121-4684-8412-9e674f13ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:28.937995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 693/1397] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:06:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:28.980830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d1532-ce1a-4830-88bf-1877bbae1c7b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:28.981325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d1532-ce1a-4830-88bf-1877bbae1c7b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:28.981450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d1532-ce1a-4830-88bf-1877bbae1c7b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:28.985276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 705/1398] 10.222.0.78 () {54 vars in 943 bytes} [Wed Aug 10 23:06:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:28.992103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fa1c-22c3-4136-8003-df20ea57b9f6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:28.992375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fa1c-22c3-4136-8003-df20ea57b9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:28.992647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fa1c-22c3-4136-8003-df20ea57b9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:29.062529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:29.070097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d450fa1c-22c3-4136-8003-df20ea57b9f6 None None] MFA Rules not processed for user `69b58d64a6e04c1487f76e5ea26c74a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:29.071337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d450fa1c-22c3-4136-8003-df20ea57b9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:29.178782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 694/1399] 10.222.0.78 () {58 vars in 1040 bytes} [Wed Aug 10 23:06:28 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:29.189432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d1532-ce1a-4830-88bf-1877bbae1c7b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:29.191003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-230d1532-ce1a-4830-88bf-1877bbae1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:29.343371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d1532-ce1a-4830-88bf-1877bbae1c7b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:29.357574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbf48-4e07-40e3-8c2a-3b003dce8196 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:29.358783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbf48-4e07-40e3-8c2a-3b003dce8196 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:29.359249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dbf48-4e07-40e3-8c2a-3b003dce8196 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:29.361569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 695/1400] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:29.367115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:29.367465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:29.367800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:29.390479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:29.439665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:29.440931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:29.462500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:29.463128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:29.463400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:29.463661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:29.465268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:29.505863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:29.545732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 696/1401] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:06:29.557133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:29.558785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed3289-3ee6-48ad-bc8b-ccfc7595aa4e service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:29.568273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 706/1402] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:06:29.591638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:29.592945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:29.714007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a87cc-22d8-49c6-80f8-fb7cd94df2df None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:29.789520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:29.830906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:29.831501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:29.831774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:29.832036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:29.833624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:29.922261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:29.923903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405b454-cfb8-445b-b376-0580eb49b3de service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:29.941977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 697/1403] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:32.338210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac751d5-54e7-442e-bbe0-3173ea5777d8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:32.338843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac751d5-54e7-442e-bbe0-3173ea5777d8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:32.338843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac751d5-54e7-442e-bbe0-3173ea5777d8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:32.340104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 707/1404] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:32.345753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15b4c-4f2f-4806-b400-038fbc9e51db None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:32.346188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15b4c-4f2f-4806-b400-038fbc9e51db None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:32.346294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15b4c-4f2f-4806-b400-038fbc9e51db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:32.416505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b2f15b4c-4f2f-4806-b400-038fbc9e51db None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:32.417732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b2f15b4c-4f2f-4806-b400-038fbc9e51db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:32.465503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:32.530666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 698/1405] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:32.576902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac751d5-54e7-442e-bbe0-3173ea5777d8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:32.578413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bac751d5-54e7-442e-bbe0-3173ea5777d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:32.659580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:32.744087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac751d5-54e7-442e-bbe0-3173ea5777d8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:32.880162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:32.880877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:32.881237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:32.881591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:32.883367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:32.918736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:32.980188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:32.982357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2d95c-e754-4848-8725-9ed8b470b4ec service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:33.001846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 708/1406] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:35.293129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84220b-81c9-4ca1-9df3-386f86a3f585 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:35.293727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84220b-81c9-4ca1-9df3-386f86a3f585 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:35.293727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84220b-81c9-4ca1-9df3-386f86a3f585 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:35.295429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 699/1407] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:35.300230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d233ba-1e80-4a7e-bce8-146930da333a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:35.300657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d233ba-1e80-4a7e-bce8-146930da333a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:35.300768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d233ba-1e80-4a7e-bce8-146930da333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:35.378127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-89d233ba-1e80-4a7e-bce8-146930da333a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:35.379375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-89d233ba-1e80-4a7e-bce8-146930da333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:35.492455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 709/1408] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:35.539641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84220b-81c9-4ca1-9df3-386f86a3f585 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:35.541152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ed84220b-81c9-4ca1-9df3-386f86a3f585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:35.709557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84220b-81c9-4ca1-9df3-386f86a3f585 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:35.732328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:35.840754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:35.841355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:35.841629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:35.841915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:35.843556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:35.954391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:35.956051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0163c-0d2a-4f47-9dc8-4986df8112a5 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:35.975813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 700/1409] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:45.490756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:45.973318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:53.488382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f42646-1520-4870-a6ac-c004e08ccbcd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:53.489283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f42646-1520-4870-a6ac-c004e08ccbcd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:53.489283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f42646-1520-4870-a6ac-c004e08ccbcd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:53.491173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 710/1410] 10.222.0.78 () {54 vars in 918 bytes} [Wed Aug 10 23:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:53.496375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42dab-87e1-4c7e-ace5-10033681d37f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:53.497644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42dab-87e1-4c7e-ace5-10033681d37f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:53.497915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42dab-87e1-4c7e-ace5-10033681d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:53.571562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c3f42dab-87e1-4c7e-ace5-10033681d37f None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:53.572774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c3f42dab-87e1-4c7e-ace5-10033681d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:53.677389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 701/1411] 10.222.0.78 () {58 vars in 1015 bytes} [Wed Aug 10 23:06:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:53.692046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fa2d9-fa6f-43ba-b7b9-007a46d05b0e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:53.692324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fa2d9-fa6f-43ba-b7b9-007a46d05b0e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:53.692594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fa2d9-fa6f-43ba-b7b9-007a46d05b0e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:53.693908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 711/1412] 10.222.0.78 () {54 vars in 918 bytes} [Wed Aug 10 23:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:53.698367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6ca5c-2b8f-4146-86d8-50a90618c1ec None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:53.698698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6ca5c-2b8f-4146-86d8-50a90618c1ec None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:53.698975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6ca5c-2b8f-4146-86d8-50a90618c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:53.769574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:53.772544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6fe6ca5c-2b8f-4146-86d8-50a90618c1ec None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:53.773759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6fe6ca5c-2b8f-4146-86d8-50a90618c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:53.888032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 702/1413] 10.222.0.78 () {58 vars in 1015 bytes} [Wed Aug 10 23:06:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:56.382911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cdd56-b133-4f00-a484-a117a1198267 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:56.383498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cdd56-b133-4f00-a484-a117a1198267 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:56.383498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cdd56-b133-4f00-a484-a117a1198267 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:56.384846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:56.390003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634df5c8-8df0-4794-b2d6-566ed9bc94c8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:56.390421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634df5c8-8df0-4794-b2d6-566ed9bc94c8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:56.390556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634df5c8-8df0-4794-b2d6-566ed9bc94c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:56.455274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-634df5c8-8df0-4794-b2d6-566ed9bc94c8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:56.456467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-634df5c8-8df0-4794-b2d6-566ed9bc94c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:56.510303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:56.550814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:56.594136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cdd56-b133-4f00-a484-a117a1198267 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:56.595698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ae1cdd56-b133-4f00-a484-a117a1198267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:56.759407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cdd56-b133-4f00-a484-a117a1198267 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:56.832026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:56.872182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:56.872785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:56.873064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:56.873329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:56.874985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:56.963148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:56.964743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e583-fbf7-4bfa-b3d1-946571e460c5 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:56.985408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 713/1416] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:57.890043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bb157-3e11-47ab-9a0e-52bad94ca3eb None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:57.890597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bb157-3e11-47ab-9a0e-52bad94ca3eb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:57.890597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bb157-3e11-47ab-9a0e-52bad94ca3eb None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:57.892049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:06:57.897213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0fe83-9d92-404b-9bb9-dcc3cb20d8bc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:57.897636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0fe83-9d92-404b-9bb9-dcc3cb20d8bc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:57.897733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0fe83-9d92-404b-9bb9-dcc3cb20d8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:57.972188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-84a0fe83-9d92-404b-9bb9-dcc3cb20d8bc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:57.973398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-84a0fe83-9d92-404b-9bb9-dcc3cb20d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:58.001429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:58.069709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:58.221361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bb157-3e11-47ab-9a0e-52bad94ca3eb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:58.222797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-552bb157-3e11-47ab-9a0e-52bad94ca3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:58.365921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bb157-3e11-47ab-9a0e-52bad94ca3eb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:58.410320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:58.509960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:58.510479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:58.510787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:58.511056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:58.513313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:58.610438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:58.612075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32c3e4-b602-41ca-ada2-836b20d0ae77 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:58.630038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 705/1419] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:06:59.544478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db695d-f577-40b9-9cf9-8ac1cdf92eef None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:59.545016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db695d-f577-40b9-9cf9-8ac1cdf92eef None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:59.545016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db695d-f577-40b9-9cf9-8ac1cdf92eef None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:59.546341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 23:06:59.551468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14824-8a4e-414b-834f-b7b4e22aaa2f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:59.551901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14824-8a4e-414b-834f-b7b4e22aaa2f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:59.552002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14824-8a4e-414b-834f-b7b4e22aaa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:59.605186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:59.626807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-31f14824-8a4e-414b-834f-b7b4e22aaa2f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:06:59.628012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-31f14824-8a4e-414b-834f-b7b4e22aaa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:59.723360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:06:59.876341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db695d-f577-40b9-9cf9-8ac1cdf92eef None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:59.877865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85db695d-f577-40b9-9cf9-8ac1cdf92eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:00.027436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db695d-f577-40b9-9cf9-8ac1cdf92eef None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:00.039932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:00.141801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:00.142441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:00.142746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:00.143017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:00.144624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:00.232738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:00.234338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ae672-9702-446d-8170-65e308dae12c service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:00.252238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 716/1422] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:01.108477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9adf4-4cc8-412d-8ad9-cf9ef780c306 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:01.109014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9adf4-4cc8-412d-8ad9-cf9ef780c306 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:01.109014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9adf4-4cc8-412d-8ad9-cf9ef780c306 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:01.110270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:01.115222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec649b58-88ff-468a-aac3-f031d44dd8e5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:01.115674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec649b58-88ff-468a-aac3-f031d44dd8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:01.115809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec649b58-88ff-468a-aac3-f031d44dd8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:01.136368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:01.192647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ec649b58-88ff-468a-aac3-f031d44dd8e5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:01.193852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec649b58-88ff-468a-aac3-f031d44dd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:01.291183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:01.443512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9adf4-4cc8-412d-8ad9-cf9ef780c306 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:01.445007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1fa9adf4-4cc8-412d-8ad9-cf9ef780c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:01.558330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:01.595935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9adf4-4cc8-412d-8ad9-cf9ef780c306 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:01.709069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:01.709577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:01.709854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:01.710122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:01.711746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:01.788892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:01.802883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:01.804480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4233d-d0c0-4725-9ad5-5d1add3cdc36 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:01.822315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 708/1425] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:02.686461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd81d6f-1708-4ab1-a1dc-502238402f85 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:02.687043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd81d6f-1708-4ab1-a1dc-502238402f85 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:02.687043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd81d6f-1708-4ab1-a1dc-502238402f85 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:02.688425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:02.693281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db873d10-4ce4-4ad0-a854-4cf246b8b3bc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:02.693711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db873d10-4ce4-4ad0-a854-4cf246b8b3bc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:02.693851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db873d10-4ce4-4ad0-a854-4cf246b8b3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:02.769753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-db873d10-4ce4-4ad0-a854-4cf246b8b3bc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:02.771000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-db873d10-4ce4-4ad0-a854-4cf246b8b3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:02.873386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:03.023650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd81d6f-1708-4ab1-a1dc-502238402f85 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:03.025123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9dd81d6f-1708-4ab1-a1dc-502238402f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:03.110813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:03.172943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd81d6f-1708-4ab1-a1dc-502238402f85 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:03.287285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:03.287806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:03.288099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:03.288368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:03.290009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:03.338101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:03.378640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:03.380717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc4af9-a13e-48cf-84d4-b91d80b5f81e service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:03.399795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 719/1428] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:04.273414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d109be-6fb7-4d77-8751-61a35470fe9a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:04.273955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d109be-6fb7-4d77-8751-61a35470fe9a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:04.273955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d109be-6fb7-4d77-8751-61a35470fe9a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:04.275218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:04.281403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e367-d03c-4f6d-a19c-9081a5b73960 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:04.281927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e367-d03c-4f6d-a19c-9081a5b73960 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:04.282039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e367-d03c-4f6d-a19c-9081a5b73960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:04.359023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-95a4e367-d03c-4f6d-a19c-9081a5b73960 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:04.360234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-95a4e367-d03c-4f6d-a19c-9081a5b73960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:04.456716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:07:04.607553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d109be-6fb7-4d77-8751-61a35470fe9a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:04.609006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f9d109be-6fb7-4d77-8751-61a35470fe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:04.659672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:04.757490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d109be-6fb7-4d77-8751-61a35470fe9a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:04.868729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:04.869335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:04.869623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:04.869882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:04.871522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:04.878876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:04.960337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:04.961960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55568225-d986-4356-a7f5-9fb63fa2f675 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:04.979542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 711/1431] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:05.938283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef98dc-f7b4-464b-b360-eff1c4613c4f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:05.938881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef98dc-f7b4-464b-b360-eff1c4613c4f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:05.938881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef98dc-f7b4-464b-b360-eff1c4613c4f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:05.940216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:05.945011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586a728-6252-402c-9d2b-4d203154d379 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:05.945457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586a728-6252-402c-9d2b-4d203154d379 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:05.945566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586a728-6252-402c-9d2b-4d203154d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:06.040029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a586a728-6252-402c-9d2b-4d203154d379 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:06.041236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a586a728-6252-402c-9d2b-4d203154d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:06.108402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:06.135967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:06.284811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef98dc-f7b4-464b-b360-eff1c4613c4f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:06.286315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5ef98dc-f7b4-464b-b360-eff1c4613c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:06.303626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:06.432856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef98dc-f7b4-464b-b360-eff1c4613c4f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:06.520554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:06.667635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:06.668230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:06.668500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:06.668764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:06.670461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:06.762616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:06.764265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4478f-e5a8-42c1-a0a7-78a714ab876d service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:06.781976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 722/1434] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:07.719460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f624-8cec-4088-ac07-c6fb68eca876 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:07.720006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f624-8cec-4088-ac07-c6fb68eca876 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:07.720006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f624-8cec-4088-ac07-c6fb68eca876 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:07.721263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:07.726352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d722a-400d-480f-884f-541e1b080f8f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:07.726778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d722a-400d-480f-884f-541e1b080f8f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:07.726927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d722a-400d-480f-884f-541e1b080f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:07.801178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d76d722a-400d-480f-884f-541e1b080f8f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:07.802373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d76d722a-400d-480f-884f-541e1b080f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:07.843430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:07.896676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:08.050077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f624-8cec-4088-ac07-c6fb68eca876 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:08.050077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9f94f624-8cec-4088-ac07-c6fb68eca876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:08.196977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f624-8cec-4088-ac07-c6fb68eca876 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:08.250703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:08.307892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:08.308844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:08.309116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:08.309380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:08.310989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:08.400676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:08.402286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471896a5-c0d9-4917-b6a1-ae2c42a58803 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:08.419941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 714/1437] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:09.350216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b6bee-bd04-4de4-828c-245499cc4140 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:09.350831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b6bee-bd04-4de4-828c-245499cc4140 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:09.350831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b6bee-bd04-4de4-828c-245499cc4140 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:09.352187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:09.357514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cad6-8b0b-496d-964e-928f1627d536 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:09.357965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cad6-8b0b-496d-964e-928f1627d536 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:09.358086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cad6-8b0b-496d-964e-928f1627d536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:09.433604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5947cad6-8b0b-496d-964e-928f1627d536 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:09.434833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5947cad6-8b0b-496d-964e-928f1627d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:09.437349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:09.531405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:09.681977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b6bee-bd04-4de4-828c-245499cc4140 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:09.683585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b34b6bee-bd04-4de4-828c-245499cc4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:09.832368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b6bee-bd04-4de4-828c-245499cc4140 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:09.860208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:09.943998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:09.944518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:09.944791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:09.945057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:09.946645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:10.043172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:10.044798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b0dbd-629b-478c-9697-50c8f7681e07 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:10.063202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 725/1440] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:11.981086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed605dc-e8c4-4827-868e-bf933285f66b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:11.981671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed605dc-e8c4-4827-868e-bf933285f66b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:11.981671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed605dc-e8c4-4827-868e-bf933285f66b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:11.983325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:11.989495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defae3c-6709-4a40-b8de-721c5f71b7d4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:11.989959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defae3c-6709-4a40-b8de-721c5f71b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:11.990061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defae3c-6709-4a40-b8de-721c5f71b7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:12.030906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:12.055649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5defae3c-6709-4a40-b8de-721c5f71b7d4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:12.056846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5defae3c-6709-4a40-b8de-721c5f71b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:12.152727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:12.190852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed605dc-e8c4-4827-868e-bf933285f66b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:12.192380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6ed605dc-e8c4-4827-868e-bf933285f66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:12.295576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed605dc-e8c4-4827-868e-bf933285f66b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:12.452935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:12.493164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:12.493754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:12.494027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:12.494296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:12.495899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:12.601112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:12.602462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c8e0a-426b-4f92-8d81-4674fad1fa27 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:12.626676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 717/1443] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:14.799303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda36f7-e519-4e05-a6ad-e443fc4c3ab0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:14.799824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda36f7-e519-4e05-a6ad-e443fc4c3ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:14.799824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda36f7-e519-4e05-a6ad-e443fc4c3ab0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:14.801185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:14.806541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755501c-0f82-4842-8117-8fb47c042742 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:14.807012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755501c-0f82-4842-8117-8fb47c042742 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:14.807113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755501c-0f82-4842-8117-8fb47c042742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:14.870472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4755501c-0f82-4842-8117-8fb47c042742 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:14.871687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4755501c-0f82-4842-8117-8fb47c042742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:14.884073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:14.965446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:15.004015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda36f7-e519-4e05-a6ad-e443fc4c3ab0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:15.005529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dfda36f7-e519-4e05-a6ad-e443fc4c3ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:15.111101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda36f7-e519-4e05-a6ad-e443fc4c3ab0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:15.123071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:15.185579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:15.186299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:15.186573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:15.186891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:15.194323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:15.297683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:15.299210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd08a1-5235-4d6f-9d39-6bc7f7db15de service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:15.317151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 728/1446] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:17.758481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655a31-7f24-48c0-80c3-028e91f0d427 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:17.759046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655a31-7f24-48c0-80c3-028e91f0d427 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:17.759046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655a31-7f24-48c0-80c3-028e91f0d427 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:17.760341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:17.765811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f78fa-0e28-41e8-9d10-94175360b75a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:17.766253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f78fa-0e28-41e8-9d10-94175360b75a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:17.766372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f78fa-0e28-41e8-9d10-94175360b75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:17.813452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:17.830148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-335f78fa-0e28-41e8-9d10-94175360b75a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:17.831362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-335f78fa-0e28-41e8-9d10-94175360b75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:17.926351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:17.965642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655a31-7f24-48c0-80c3-028e91f0d427 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:17.967744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e7655a31-7f24-48c0-80c3-028e91f0d427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:18.067659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655a31-7f24-48c0-80c3-028e91f0d427 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:18.105621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:18.140878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:18.141467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:18.141742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:18.142007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:18.143621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:18.252818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:18.254253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159a2d6-180d-47bb-9c6d-e6802dde42c8 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:18.272105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 720/1449] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:20.499965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa438e4-15cf-4f9c-a624-817b0ec6965e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:20.500510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa438e4-15cf-4f9c-a624-817b0ec6965e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:20.500510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa438e4-15cf-4f9c-a624-817b0ec6965e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:20.501856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:20.507430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f42e7-6440-4484-85d5-6f813ceae729 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:20.507862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f42e7-6440-4484-85d5-6f813ceae729 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:20.507978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f42e7-6440-4484-85d5-6f813ceae729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:20.572087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bc7f42e7-6440-4484-85d5-6f813ceae729 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:20.573283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bc7f42e7-6440-4484-85d5-6f813ceae729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:20.593784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:20.668080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:20.707842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa438e4-15cf-4f9c-a624-817b0ec6965e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:20.709367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2aa438e4-15cf-4f9c-a624-817b0ec6965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:20.812351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa438e4-15cf-4f9c-a624-817b0ec6965e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:20.831852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:20.885808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:20.886413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:20.886711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:20.886986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:20.888587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:20.996071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:20.997429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126bc96-8be9-4702-a083-a18023678c54 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:21.015766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 731/1452] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:23.065274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676336-19af-43e5-962f-49368fb4b976 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:23.066097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676336-19af-43e5-962f-49368fb4b976 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:23.066097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676336-19af-43e5-962f-49368fb4b976 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:23.067646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:23.072754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de782-2555-4761-aa36-103f84b6c75e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:23.073200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de782-2555-4761-aa36-103f84b6c75e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:23.073324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de782-2555-4761-aa36-103f84b6c75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:23.137051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:23.140219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-292de782-2555-4761-aa36-103f84b6c75e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:23.141448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-292de782-2555-4761-aa36-103f84b6c75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:23.237332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:23.276684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f676336-19af-43e5-962f-49368fb4b976 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:23.278199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8f676336-19af-43e5-962f-49368fb4b976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:23.379740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f676336-19af-43e5-962f-49368fb4b976 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:23.429766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:23.454737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:23.455326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:23.455659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:23.455917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:23.457484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:23.565104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:23.566525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa2cef-41c4-4f07-ac47-9a7b4107c3c9 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:23.584602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 723/1455] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:25.666855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231a28-360c-45ab-83cf-ba70dd1e2e1c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:25.667423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231a28-360c-45ab-83cf-ba70dd1e2e1c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:25.667423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231a28-360c-45ab-83cf-ba70dd1e2e1c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:25.668817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:25.674931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf47da1-c3e5-4dc3-a106-9c272a1a36fd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:25.675467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf47da1-c3e5-4dc3-a106-9c272a1a36fd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:25.675467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf47da1-c3e5-4dc3-a106-9c272a1a36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:25.739784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6cf47da1-c3e5-4dc3-a106-9c272a1a36fd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:25.740982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6cf47da1-c3e5-4dc3-a106-9c272a1a36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:25.772776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:25.839531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:25.877594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231a28-360c-45ab-83cf-ba70dd1e2e1c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:25.883480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-43231a28-360c-45ab-83cf-ba70dd1e2e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:25.986295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231a28-360c-45ab-83cf-ba70dd1e2e1c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:26.015489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:26.058818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:26.059436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:26.059716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:26.060003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:26.062082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:26.169722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:26.171678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882c65e-5d31-45c1-9dfd-aab5c2815601 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:26.189805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 734/1458] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:28.260473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75f55-02f4-427f-a137-e7eb13808b8a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:28.261049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75f55-02f4-427f-a137-e7eb13808b8a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:28.261049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75f55-02f4-427f-a137-e7eb13808b8a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:28.262432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:28.268864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e2ff4-afec-4a5a-b934-1506e81b8db0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:28.269393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e2ff4-afec-4a5a-b934-1506e81b8db0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:28.269811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e2ff4-afec-4a5a-b934-1506e81b8db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:28.344477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-397e2ff4-afec-4a5a-b934-1506e81b8db0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:28.345703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-397e2ff4-afec-4a5a-b934-1506e81b8db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:28.353894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:28.443844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:28.482576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75f55-02f4-427f-a137-e7eb13808b8a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:28.484274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9be75f55-02f4-427f-a137-e7eb13808b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:28.587203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75f55-02f4-427f-a137-e7eb13808b8a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:28.644219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:28.663326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:28.663907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:28.664180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:28.664442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:28.666024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:28.771544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:28.772974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c0a6c-0296-460c-87da-5df5dacc4089 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:28.791921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 726/1461] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:07:30.856636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784307fa-e253-4234-b907-ef0b7bd0d35c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:30.857665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784307fa-e253-4234-b907-ef0b7bd0d35c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:30.857765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784307fa-e253-4234-b907-ef0b7bd0d35c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:30.859071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:30.864221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aacaf-bb90-41aa-86e4-73bd90d02374 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:30.864609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aacaf-bb90-41aa-86e4-73bd90d02374 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:30.864775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aacaf-bb90-41aa-86e4-73bd90d02374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:30.928597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4b9aacaf-bb90-41aa-86e4-73bd90d02374 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:30.929792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4b9aacaf-bb90-41aa-86e4-73bd90d02374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:30.967810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:31.027156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:31.065223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784307fa-e253-4234-b907-ef0b7bd0d35c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:31.066823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-784307fa-e253-4234-b907-ef0b7bd0d35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:31.169177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784307fa-e253-4234-b907-ef0b7bd0d35c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:31.206281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:31.242818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:31.243428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:31.243704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:31.243973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:31.245571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:31.357789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:31.359171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61c2a9-c917-40e1-b358-c8242ade2d55 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:31.378337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 737/1464] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:33.398764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcc60b-2c5b-4cca-abc0-e0418e750025 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:33.399364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcc60b-2c5b-4cca-abc0-e0418e750025 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:33.399364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcc60b-2c5b-4cca-abc0-e0418e750025 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:33.400702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:33.406381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089f932-7af1-4a1d-b8e5-0645e98f3877 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:33.406854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089f932-7af1-4a1d-b8e5-0645e98f3877 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:33.406958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089f932-7af1-4a1d-b8e5-0645e98f3877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:33.471437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8089f932-7af1-4a1d-b8e5-0645e98f3877 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:33.472628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8089f932-7af1-4a1d-b8e5-0645e98f3877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:33.495432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:33.577637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:33.615198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcc60b-2c5b-4cca-abc0-e0418e750025 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:33.616751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e3fcc60b-2c5b-4cca-abc0-e0418e750025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:33.719909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcc60b-2c5b-4cca-abc0-e0418e750025 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:33.787955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:33.795138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:33.795732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:33.796002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:33.796265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:33.797875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:33.906054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:33.907525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f42ee-abc0-4f7f-9c57-aae302836210 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:33.925320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 729/1467] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:35.931385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b15383-4ccf-4628-9333-f964f0a6ba3c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:35.931960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b15383-4ccf-4628-9333-f964f0a6ba3c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:35.931960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b15383-4ccf-4628-9333-f964f0a6ba3c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:35.933259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:35.939017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb4fc-f865-4c51-83b0-b36294f0c0ba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:35.939473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb4fc-f865-4c51-83b0-b36294f0c0ba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:35.939582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb4fc-f865-4c51-83b0-b36294f0c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:36.005607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-43eeb4fc-f865-4c51-83b0-b36294f0c0ba None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:36.006840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-43eeb4fc-f865-4c51-83b0-b36294f0c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:36.053866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:36.101105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:36.138497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b15383-4ccf-4628-9333-f964f0a6ba3c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:36.140098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-77b15383-4ccf-4628-9333-f964f0a6ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:36.241389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b15383-4ccf-4628-9333-f964f0a6ba3c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:36.288294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:36.314630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:36.315255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:36.315532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:36.315799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:36.317452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:36.426503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:36.427999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86e4d-4985-4579-8009-e414f35de75b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:36.446514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 740/1470] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:46.099422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=22 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:46.444205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=66 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:56.415201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c20e6-371c-4753-8ae8-c27db5915417 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:56.415790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c20e6-371c-4753-8ae8-c27db5915417 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:56.415790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c20e6-371c-4753-8ae8-c27db5915417 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:56.417103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 731/1471] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:56.423031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbc9e5-c5f1-4261-981f-7ba876e83a48 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:56.423492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbc9e5-c5f1-4261-981f-7ba876e83a48 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:56.423611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbc9e5-c5f1-4261-981f-7ba876e83a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:56.498626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-20bbc9e5-c5f1-4261-981f-7ba876e83a48 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:56.499869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-20bbc9e5-c5f1-4261-981f-7ba876e83a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:56.593789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 741/1472] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:56.679473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc533-8dbd-4918-b681-05ccff5d43b3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:56.679758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc533-8dbd-4918-b681-05ccff5d43b3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:56.680051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc533-8dbd-4918-b681-05ccff5d43b3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:56.681372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 732/1473] 10.222.0.78 () {54 vars in 967 bytes} [Wed Aug 10 23:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:56.686223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b10d-018c-4584-9e3a-e5aa44fa205b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:56.686530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b10d-018c-4584-9e3a-e5aa44fa205b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:56.686826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b10d-018c-4584-9e3a-e5aa44fa205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:56.765041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3fc5b10d-018c-4584-9e3a-e5aa44fa205b None None] MFA Rules not processed for user `f28dc00f9c744cc4b103b8e01d65f75f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:56.766260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3fc5b10d-018c-4584-9e3a-e5aa44fa205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:56.787604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:56.843249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 742/1474] 10.222.0.78 () {58 vars in 1064 bytes} [Wed Aug 10 23:07:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:56.850684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fc533-8dbd-4918-b681-05ccff5d43b3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:56.852232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-097fc533-8dbd-4918-b681-05ccff5d43b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:56.954022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fc533-8dbd-4918-b681-05ccff5d43b3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:57.026375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:57.027001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:57.027272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:57.027538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:57.029815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:57.086365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:57.140717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:57.142154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4e98e-beac-469e-a178-541cfab9d657 service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:57.161696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 733/1475] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:59.245529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288f02-6663-49a3-99a2-a7e003b2af15 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:59.246130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288f02-6663-49a3-99a2-a7e003b2af15 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:59.246130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288f02-6663-49a3-99a2-a7e003b2af15 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:59.248010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 743/1476] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:07:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:07:59.253268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f0a9a-3f4f-4d20-8fef-0ec66e28e444 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:59.253665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f0a9a-3f4f-4d20-8fef-0ec66e28e444 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:59.253827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f0a9a-3f4f-4d20-8fef-0ec66e28e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:59.317854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c74f0a9a-3f4f-4d20-8fef-0ec66e28e444 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:59.319071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c74f0a9a-3f4f-4d20-8fef-0ec66e28e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:59.416507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 734/1477] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:59.440436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288f02-6663-49a3-99a2-a7e003b2af15 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:59.441969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-55288f02-6663-49a3-99a2-a7e003b2af15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:59.565990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288f02-6663-49a3-99a2-a7e003b2af15 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:59.614225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:59.659867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:59.660471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:59.660748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:59.661019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:59.661651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:59.663399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:07:59.663399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:07:59.663399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:07:59.664588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28791d3f-8033-4c5a-85ab-f1548f6bfc02 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:59.684433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 744/1478] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 10 23:07:59 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:08:01.709205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747af0-c537-4523-bd43-33376776988d None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:01.709744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747af0-c537-4523-bd43-33376776988d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:01.709744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747af0-c537-4523-bd43-33376776988d None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:01.711107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 735/1479] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:08:01.716856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09885b8-5343-4356-8e61-6513c1552e0d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:01.717270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09885b8-5343-4356-8e61-6513c1552e0d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:01.717462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09885b8-5343-4356-8e61-6513c1552e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:01.783966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f09885b8-5343-4356-8e61-6513c1552e0d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:08:01.785182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f09885b8-5343-4356-8e61-6513c1552e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:01.884379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 745/1480] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:08:01.927316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57747af0-c537-4523-bd43-33376776988d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:08:01.928824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57747af0-c537-4523-bd43-33376776988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:01.938918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:02.074032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57747af0-c537-4523-bd43-33376776988d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:08:02.155993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:02.189818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:08:02.190507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:02.190860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:02.191174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:02.192851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:02.282604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:08:02.284289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa65e9-68c2-4947-8e38-f8c9432414ce service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:08:02.302844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 736/1481] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:08:04.488395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75132df-e191-4992-9117-e3492ce268af None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:04.488985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75132df-e191-4992-9117-e3492ce268af None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:04.488985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75132df-e191-4992-9117-e3492ce268af None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:04.490326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 746/1482] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:08:04.495720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42807e4c-3cdb-4a52-a9d8-ce1639f8f9e2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:04.496200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42807e4c-3cdb-4a52-a9d8-ce1639f8f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:04.496312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42807e4c-3cdb-4a52-a9d8-ce1639f8f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:04.562371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-42807e4c-3cdb-4a52-a9d8-ce1639f8f9e2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:08:04.563600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-42807e4c-3cdb-4a52-a9d8-ce1639f8f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:04.609337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:04.659125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 737/1483] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:08:04.695094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75132df-e191-4992-9117-e3492ce268af None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:08:04.696726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a75132df-e191-4992-9117-e3492ce268af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:04.720766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:04.799576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75132df-e191-4992-9117-e3492ce268af None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:08:04.870806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:08:04.871905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:04.872173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:04.872433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:04.874004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:04.945817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:04.980984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:08:04.982434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6348f-657e-4942-8cf5-f66190324c3b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:08:05.000548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 747/1484] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:08:07.088655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1871e6c-d02c-4424-839c-0d7e5aa8966b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:07.089193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1871e6c-d02c-4424-839c-0d7e5aa8966b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:07.089193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1871e6c-d02c-4424-839c-0d7e5aa8966b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:07.091016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 738/1485] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:08:07.096436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e68a9d-376d-4814-943e-d614eb4c54c3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:07.096953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e68a9d-376d-4814-943e-d614eb4c54c3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:07.097059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e68a9d-376d-4814-943e-d614eb4c54c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:07.164638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f9e68a9d-376d-4814-943e-d614eb4c54c3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:08:07.165834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f9e68a9d-376d-4814-943e-d614eb4c54c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:07.262840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 748/1486] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:08:07.301456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1871e6c-d02c-4424-839c-0d7e5aa8966b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:08:07.303100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a1871e6c-d02c-4424-839c-0d7e5aa8966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:07.404891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1871e6c-d02c-4424-839c-0d7e5aa8966b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:08:07.475370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:07.477138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:08:07.477723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:07.477992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:07.478256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:07.479858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:07.586039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:08:07.587504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74a6b0-dac8-4917-9895-aec1a747cbb8 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:08:07.605663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 739/1487] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:08:09.736031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c80647-ae6d-4b13-97cb-dcd8f988bd92 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:09.736580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c80647-ae6d-4b13-97cb-dcd8f988bd92 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:09.736580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c80647-ae6d-4b13-97cb-dcd8f988bd92 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:09.737909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 749/1488] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:08:09.743098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c198f4-9b68-421e-b3d9-75d0525d1920 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:09.743517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c198f4-9b68-421e-b3d9-75d0525d1920 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:09.743623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c198f4-9b68-421e-b3d9-75d0525d1920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:09.808434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-84c198f4-9b68-421e-b3d9-75d0525d1920 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:08:09.810160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-84c198f4-9b68-421e-b3d9-75d0525d1920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:09.863650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:09.906996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 740/1489] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:08:09.948474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c80647-ae6d-4b13-97cb-dcd8f988bd92 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:08:09.950089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a9c80647-ae6d-4b13-97cb-dcd8f988bd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:09.987382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:10.054254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c80647-ae6d-4b13-97cb-dcd8f988bd92 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:08:10.127757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:08:10.128359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:10.128639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:10.128911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:10.130509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:10.213936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:10.244426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:08:10.245774 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f215923-ebf1-414c-89af-e835fe84db07 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:08:10.264099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 750/1490] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:08:12.703140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1668d07-db12-4564-801c-542b0ead6730 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:12.703707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1668d07-db12-4564-801c-542b0ead6730 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:12.703707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1668d07-db12-4564-801c-542b0ead6730 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:12.704927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 741/1491] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 10 23:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:08:12.710043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24877b9-1175-4363-b362-c169c79fae1f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:12.710476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24877b9-1175-4363-b362-c169c79fae1f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:12.710636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24877b9-1175-4363-b362-c169c79fae1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:12.776265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a24877b9-1175-4363-b362-c169c79fae1f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:08:12.778038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a24877b9-1175-4363-b362-c169c79fae1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:12.878730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 751/1492] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 10 23:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:08:12.918252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1668d07-db12-4564-801c-542b0ead6730 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:08:12.919912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b1668d07-db12-4564-801c-542b0ead6730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:13.023997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1668d07-db12-4564-801c-542b0ead6730 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:08:13.096049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:08:13.096649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:13.096927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:13.097198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:13.098946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:08:13.115216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:13.215804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:08:13.217271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc88be-c345-4cbb-9226-b81af3361734 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:08:13.235066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 742/1493] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:08:22.876891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=82 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:23.232813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=50 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:53.943453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:53.944045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:53.944045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:54.026190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:09:54.027435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:54.121603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 752/1494] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:09:54.129878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:54.130394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:54.130512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:54.206146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:09:54.207300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:54.305221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 743/1495] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:09:54.316673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:54.317929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:54.330294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356a76c-fa32-4eed-a453-19cf43b87d7e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:54.391917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:54.424786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:54.425912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:54.425912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:54.425912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:54.426532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:09:54.426879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:54.427897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:54.427897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:09:54.427897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:09:54.428785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:54.440732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 753/1496] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:09:54.449686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:54.458727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374092b3-5942-4c13-bd55-1d61cb7e3244 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:54.520117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:54.552473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:54.553067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:54.553347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:54.553618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:54.554226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:54.555993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:54.555993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:09:54.555993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:09:54.557807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:54.692883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 744/1497] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:09:54 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:09:54.697287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:54.698875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:54.732467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23019d87-b6be-4fd5-aeb5-6dcf3eaeb6ee admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:54.825343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:54.825994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:54.826270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:54.826538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:54.827230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:54.828232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:54.828232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:09:54.828232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:09:54.829454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:54.937521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 754/1498] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:09:54 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:09:54.941667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:54.943215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:55.003795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112082-f504-45ef-95bb-451a62cb5d4f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:55.085957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:55.096622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:55.097220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:55.097498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:55.097769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:55.098415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:55.099464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:55.099464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:09:55.099464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:09:55.100361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:55.113288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 745/1499] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:09:55.117366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:55.118930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:55.130276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526b490-700d-4745-9e8e-f38af284ed6e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:55.155594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:55.225769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:55.226379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:55.226939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:55.227223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:55.227865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:55.229434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:55.229434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:09:55.229434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:09:55.230323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626f5bb-38a1-41d3-95aa-ad8cff8bbddf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:55.242731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 755/1500] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:09:55.247075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:55.249033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:55.290033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd2a03-5e3b-446e-8812-12026cd95818 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:55.381754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:55.382391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:55.382675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:55.382949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] PATH_INFO: `/v3/projects/1ed4b2111a3f40daa07518479d11271d/users/d6e935828b324f39a43ab02f1c02cf4e/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:55.405948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed4b2111a3f40daa07518479d11271d, user_id=d6e935828b324f39a43ab02f1c02cf4e, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:55.407001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:55.407001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:09:55.407001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:09:55.408919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b42b3-a52d-48be-b7fd-41aecaf36828 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:55.495429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 746/1501] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:09:55 2022] PUT /identity/v3/projects/1ed4b2111a3f40daa07518479d11271d/users/d6e935828b324f39a43ab02f1c02cf4e/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:09:55.507562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295b21-242f-416a-a27d-9df51c72497c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:55.507845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295b21-242f-416a-a27d-9df51c72497c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:55.508115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295b21-242f-416a-a27d-9df51c72497c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:55.564869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2d295b21-242f-416a-a27d-9df51c72497c None None] MFA Rules not processed for user `d6e935828b324f39a43ab02f1c02cf4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:09:55.565993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2d295b21-242f-416a-a27d-9df51c72497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:55.642165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 756/1502] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:09:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:09:55.656024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457e295-f1c7-4f6f-9904-d8b95485450e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:55.656312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457e295-f1c7-4f6f-9904-d8b95485450e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:55.656577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457e295-f1c7-4f6f-9904-d8b95485450e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:55.657916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 747/1503] 10.222.0.78 () {54 vars in 964 bytes} [Wed Aug 10 23:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:09:55.662746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626496-55a1-4a8b-94a0-eb554cc62601 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:55.663038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626496-55a1-4a8b-94a0-eb554cc62601 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:55.663314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626496-55a1-4a8b-94a0-eb554cc62601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:55.700706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:55.744629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-93626496-55a1-4a8b-94a0-eb554cc62601 None None] MFA Rules not processed for user `54fff64dfcf8434d867c028a72c17a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:09:55.745850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-93626496-55a1-4a8b-94a0-eb554cc62601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:55.820478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 757/1504] 10.222.0.78 () {58 vars in 1061 bytes} [Wed Aug 10 23:09:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:09:55.829230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457e295-f1c7-4f6f-9904-d8b95485450e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:55.830757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5457e295-f1c7-4f6f-9904-d8b95485450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:55.879591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:55.932818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457e295-f1c7-4f6f-9904-d8b95485450e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:56.004288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:56.004964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:56.005237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:56.005497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:56.007096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:56.104258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:56.105703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:56.109117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:56.123911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 748/1505] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:09:56.168094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fd59-3e57-4774-bc90-8d89d2f9a147 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:56.168494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fd59-3e57-4774-bc90-8d89d2f9a147 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:56.168644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fd59-3e57-4774-bc90-8d89d2f9a147 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:56.169897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 758/1506] 10.222.0.78 () {52 vars in 830 bytes} [Wed Aug 10 23:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:09:56.173470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:56.174691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:56.276682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddfb0b-1581-46f1-ac49-6dca530ffee2 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:56.368159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:56.368830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:56.369584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:56.369847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] PATH_INFO: `/v3/users/d6e935828b324f39a43ab02f1c02cf4e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:56.381730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e935828b324f39a43ab02f1c02cf4e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:56.382786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:56.382786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:09:56.382786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:09:56.383971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:56.393477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:56.634401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e935828b324f39a43ab02f1c02cf4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:56.657196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e935828b324f39a43ab02f1c02cf4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:56.671155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e935828b324f39a43ab02f1c02cf4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:56.671610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e935828b324f39a43ab02f1c02cf4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:56.672126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e935828b324f39a43ab02f1c02cf4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:56.742890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 749/1507] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:09:56 2022] DELETE /identity/v3/users/d6e935828b324f39a43ab02f1c02cf4e => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:09:56.754112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:56.754391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:56.754681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:56.811567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:09:56.812764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:56.888658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:56.901371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 759/1508] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:09:56.911186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:56.912470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:57.011675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ebf38-e998-49e6-908f-0d19dd73e472 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:57.079158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:57.103119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:57.103725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:57.103997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:57.104275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:57.105859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:57.215994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:57.217347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd8f5b-4f1a-4c94-b365-cf0ba31bd9a9 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:57.235597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 750/1509] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:09:57.720571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:57.721863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:57.733832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc242a5-7862-4138-8500-d06733f77245 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:57.837233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:57.838567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:57.838876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:57.839149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] PATH_INFO: `/v3/projects/1ed4b2111a3f40daa07518479d11271d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:57.849221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ed4b2111a3f40daa07518479d11271d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:57.850249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:09:57.850249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:09:57.850249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:09:57.851533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:57.988445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed4b2111a3f40daa07518479d11271d', 'request_id': 'req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:58.074794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed4b2111a3f40daa07518479d11271d', 'request_id': 'req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:58.081988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed4b2111a3f40daa07518479d11271d', 'request_id': 'req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:58.113741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] The token cache is being invalidate because project 1ed4b2111a3f40daa07518479d11271d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:09:58.114028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:58.114464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-93dbcc9e-a8b4-4e1e-9e81-e0bfb07215ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:09:58.115941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 760/1510] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:09:57 2022] DELETE /identity/v3/projects/1ed4b2111a3f40daa07518479d11271d => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:10:07.233306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=62 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:10:08.111024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:11:25.290048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf6d1c-2343-40ac-a241-e3e9e2101654 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:25.290613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf6d1c-2343-40ac-a241-e3e9e2101654 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:25.290613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf6d1c-2343-40ac-a241-e3e9e2101654 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:25.292169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:25.297025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bdb26-8174-4635-a4dc-6b5efc01c270 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:25.297442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bdb26-8174-4635-a4dc-6b5efc01c270 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:25.297575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bdb26-8174-4635-a4dc-6b5efc01c270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:25.377924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-599bdb26-8174-4635-a4dc-6b5efc01c270 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:25.379687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-599bdb26-8174-4635-a4dc-6b5efc01c270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:25.481694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:11:25.636066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf6d1c-2343-40ac-a241-e3e9e2101654 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:25.638148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-14cf6d1c-2343-40ac-a241-e3e9e2101654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:25.754801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf6d1c-2343-40ac-a241-e3e9e2101654 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:25.760509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b8834-672a-4035-a7d6-23822037a607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:25.761107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b8834-672a-4035-a7d6-23822037a607 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:25.761412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b8834-672a-4035-a7d6-23822037a607 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:25.761687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b8834-672a-4035-a7d6-23822037a607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:25.769768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b8834-672a-4035-a7d6-23822037a607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:25.771410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b8834-672a-4035-a7d6-23822037a607 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:25.780584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 752/1513] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:26.614502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043698d7-8579-48ae-91ef-8e741fe5e510 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:26.615064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043698d7-8579-48ae-91ef-8e741fe5e510 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:26.615064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043698d7-8579-48ae-91ef-8e741fe5e510 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:26.616432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:26.621754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c169ca-2968-4af1-b4fa-d9e2dd92e8a7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:26.622796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c169ca-2968-4af1-b4fa-d9e2dd92e8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:26.622796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c169ca-2968-4af1-b4fa-d9e2dd92e8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:26.641998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d9c169ca-2968-4af1-b4fa-d9e2dd92e8a7 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:26.643235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d9c169ca-2968-4af1-b4fa-d9e2dd92e8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:26.651570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:26.804000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043698d7-8579-48ae-91ef-8e741fe5e510 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:26.818736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043698d7-8579-48ae-91ef-8e741fe5e510 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:26.824083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfefbe6-8156-4b72-8b36-fb8e0fc3aeee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:26.824695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfefbe6-8156-4b72-8b36-fb8e0fc3aeee service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:26.824974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfefbe6-8156-4b72-8b36-fb8e0fc3aeee service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:26.825246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfefbe6-8156-4b72-8b36-fb8e0fc3aeee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:26.833788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfefbe6-8156-4b72-8b36-fb8e0fc3aeee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:26.835407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfefbe6-8156-4b72-8b36-fb8e0fc3aeee service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:26.844593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 763/1516] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:27.385664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958575-fc2b-444e-9bef-8ebba2ffb939 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:27.385664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958575-fc2b-444e-9bef-8ebba2ffb939 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:27.385664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958575-fc2b-444e-9bef-8ebba2ffb939 None None] PATH_INFO: `/v3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:27.385664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 754/1517] 10.222.0.89 () {54 vars in 925 bytes} [Wed Aug 10 23:11:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:11:27.391725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d94bd-9e04-405b-8b43-87826999c202 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:27.392028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d94bd-9e04-405b-8b43-87826999c202 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:27.392295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d94bd-9e04-405b-8b43-87826999c202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:27.453759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5f3d94bd-9e04-405b-8b43-87826999c202 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:27.455193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5f3d94bd-9e04-405b-8b43-87826999c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:27.473005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:11:27.532929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 764/1518] 10.222.0.89 () {58 vars in 1015 bytes} [Wed Aug 10 23:11:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:31.599372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db84db-0b5a-4e25-98d6-6d672e65fa31 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:31.599908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db84db-0b5a-4e25-98d6-6d672e65fa31 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:31.599908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db84db-0b5a-4e25-98d6-6d672e65fa31 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:31.601176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 755/1519] 10.222.0.78 () {54 vars in 961 bytes} [Wed Aug 10 23:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:31.606143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25313-bc30-4678-9a9e-c1ca31ff34a2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:31.606606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25313-bc30-4678-9a9e-c1ca31ff34a2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:31.606751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25313-bc30-4678-9a9e-c1ca31ff34a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:31.625455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1bb25313-bc30-4678-9a9e-c1ca31ff34a2 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:31.626665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1bb25313-bc30-4678-9a9e-c1ca31ff34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:31.635540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 765/1520] 10.222.0.78 () {58 vars in 1058 bytes} [Wed Aug 10 23:11:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:31.643093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db84db-0b5a-4e25-98d6-6d672e65fa31 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:31.657065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db84db-0b5a-4e25-98d6-6d672e65fa31 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:31.661952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b59d5-8026-4368-8e5f-0d07b0fc1e80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:31.662526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b59d5-8026-4368-8e5f-0d07b0fc1e80 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:31.662833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b59d5-8026-4368-8e5f-0d07b0fc1e80 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:31.663105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b59d5-8026-4368-8e5f-0d07b0fc1e80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:31.672229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b59d5-8026-4368-8e5f-0d07b0fc1e80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:31.673812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b59d5-8026-4368-8e5f-0d07b0fc1e80 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:31.683777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 756/1521] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:38.087211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04fa5e-e310-41b2-8da2-070db4a746cc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:38.087741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04fa5e-e310-41b2-8da2-070db4a746cc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:38.087741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04fa5e-e310-41b2-8da2-070db4a746cc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:38.089029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:38.093832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c5e2-8fb0-481d-9e01-9f4208a54185 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:38.094275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c5e2-8fb0-481d-9e01-9f4208a54185 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:38.094383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c5e2-8fb0-481d-9e01-9f4208a54185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:38.113311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b251c5e2-8fb0-481d-9e01-9f4208a54185 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:38.114730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b251c5e2-8fb0-481d-9e01-9f4208a54185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:38.122452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:38.274519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04fa5e-e310-41b2-8da2-070db4a746cc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:38.288739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04fa5e-e310-41b2-8da2-070db4a746cc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:38.298889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151acfc-5360-4147-bf91-c010f0b27922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:38.298889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151acfc-5360-4147-bf91-c010f0b27922 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:38.298889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151acfc-5360-4147-bf91-c010f0b27922 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:38.298889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151acfc-5360-4147-bf91-c010f0b27922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:38.303218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151acfc-5360-4147-bf91-c010f0b27922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:38.304794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151acfc-5360-4147-bf91-c010f0b27922 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:38.314436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 767/1524] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:39.141589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d6373-b15b-4ab8-9fe8-6872b7f68a6f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:39.142129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d6373-b15b-4ab8-9fe8-6872b7f68a6f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:39.142129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d6373-b15b-4ab8-9fe8-6872b7f68a6f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:39.143418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 23:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:39.148457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40529f-b75e-4c2b-b96e-ec8351e02243 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:39.148847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40529f-b75e-4c2b-b96e-ec8351e02243 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:39.148998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40529f-b75e-4c2b-b96e-ec8351e02243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:39.167663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-aa40529f-b75e-4c2b-b96e-ec8351e02243 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:39.168863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aa40529f-b75e-4c2b-b96e-ec8351e02243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:39.176604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 23:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:39.329819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d6373-b15b-4ab8-9fe8-6872b7f68a6f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:39.343969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d6373-b15b-4ab8-9fe8-6872b7f68a6f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:39.349604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4e106-a395-477e-bba5-8d801a7efd35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:39.350188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e106-a395-477e-bba5-8d801a7efd35 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:39.350465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e106-a395-477e-bba5-8d801a7efd35 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:39.350767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e106-a395-477e-bba5-8d801a7efd35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:39.358913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4e106-a395-477e-bba5-8d801a7efd35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:39.360488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4e106-a395-477e-bba5-8d801a7efd35 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:39.370324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 759/1527] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:43.258866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f05ac-8d87-4f50-abf2-4e90d19ef7a5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:43.259454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f05ac-8d87-4f50-abf2-4e90d19ef7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:43.259454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f05ac-8d87-4f50-abf2-4e90d19ef7a5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:43.260729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:43.265979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ea10f-b45e-40d5-876b-03dfb2c37c44 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:43.266438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ea10f-b45e-40d5-876b-03dfb2c37c44 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:43.266548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ea10f-b45e-40d5-876b-03dfb2c37c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:43.285882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-999ea10f-b45e-40d5-876b-03dfb2c37c44 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:43.287102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-999ea10f-b45e-40d5-876b-03dfb2c37c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:43.294854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:43.369082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f05ac-8d87-4f50-abf2-4e90d19ef7a5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:43.371034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca2f05ac-8d87-4f50-abf2-4e90d19ef7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:43.449662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f05ac-8d87-4f50-abf2-4e90d19ef7a5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:43.453158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281c2be-8820-4c07-9622-6ebf74bfad95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:43.453754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281c2be-8820-4c07-9622-6ebf74bfad95 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:43.454025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281c2be-8820-4c07-9622-6ebf74bfad95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:43.454288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281c2be-8820-4c07-9622-6ebf74bfad95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:43.463419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281c2be-8820-4c07-9622-6ebf74bfad95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:43.464822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281c2be-8820-4c07-9622-6ebf74bfad95 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:43.474279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 770/1530] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:45.445388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5191f-6636-46a1-94ea-2bff8cb7ed94 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:45.445896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5191f-6636-46a1-94ea-2bff8cb7ed94 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:45.446024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5191f-6636-46a1-94ea-2bff8cb7ed94 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:45.447345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:45.452510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bf89-9f34-477f-bb92-ac87c2dfd96c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:45.452940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bf89-9f34-477f-bb92-ac87c2dfd96c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:45.453048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bf89-9f34-477f-bb92-ac87c2dfd96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:45.471854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d3b6bf89-9f34-477f-bb92-ac87c2dfd96c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:45.473035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d3b6bf89-9f34-477f-bb92-ac87c2dfd96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:45.481521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:45.517709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5191f-6636-46a1-94ea-2bff8cb7ed94 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:45.528303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5191f-6636-46a1-94ea-2bff8cb7ed94 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:45.531720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf60a81-a487-4401-b8e9-0ad7c7ffe700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:45.532293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf60a81-a487-4401-b8e9-0ad7c7ffe700 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:45.532567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf60a81-a487-4401-b8e9-0ad7c7ffe700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:45.532839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf60a81-a487-4401-b8e9-0ad7c7ffe700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:45.542019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf60a81-a487-4401-b8e9-0ad7c7ffe700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:45.543486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf60a81-a487-4401-b8e9-0ad7c7ffe700 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:45.552599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 762/1533] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:49.400486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c899-32e9-432f-a2b4-8d13d68195be None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:49.401076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c899-32e9-432f-a2b4-8d13d68195be None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:49.401180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c899-32e9-432f-a2b4-8d13d68195be None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:49.402443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:49.407543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418811-4de7-40be-897a-0d2c0704e51d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:49.407994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418811-4de7-40be-897a-0d2c0704e51d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:49.408112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418811-4de7-40be-897a-0d2c0704e51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:49.415590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:11:49.429723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-46418811-4de7-40be-897a-0d2c0704e51d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:49.429864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-46418811-4de7-40be-897a-0d2c0704e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:49.437994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:49.748735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c899-32e9-432f-a2b4-8d13d68195be None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:49.750484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3bc8c899-32e9-432f-a2b4-8d13d68195be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:49.814664 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:11:49.827804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c899-32e9-432f-a2b4-8d13d68195be None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:49.831473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad348f-acd7-44cf-bc68-2d6443295018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:49.831991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad348f-acd7-44cf-bc68-2d6443295018 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:49.832274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad348f-acd7-44cf-bc68-2d6443295018 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:49.832543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad348f-acd7-44cf-bc68-2d6443295018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:49.842318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad348f-acd7-44cf-bc68-2d6443295018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:49.843791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad348f-acd7-44cf-bc68-2d6443295018 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:49.853774 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 773/1536] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:52.001742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534242e5-5624-4f07-a277-2bc86ce19bec None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:52.002342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534242e5-5624-4f07-a277-2bc86ce19bec None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:52.002342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534242e5-5624-4f07-a277-2bc86ce19bec None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:52.003685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 23:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:11:52.010452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9baa8-962f-44f9-9921-ea0f15c931d6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:52.013813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9baa8-962f-44f9-9921-ea0f15c931d6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:52.014107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9baa8-962f-44f9-9921-ea0f15c931d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:52.033931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0ab9baa8-962f-44f9-9921-ea0f15c931d6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:11:52.035398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ab9baa8-962f-44f9-9921-ea0f15c931d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:52.047266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 23:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:11:52.354342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534242e5-5624-4f07-a277-2bc86ce19bec None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:11:52.356058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-534242e5-5624-4f07-a277-2bc86ce19bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:11:52.369149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534242e5-5624-4f07-a277-2bc86ce19bec None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:11:52.374258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7356fa9-c77d-46cc-b446-e9fc7a36a289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:11:52.374900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7356fa9-c77d-46cc-b446-e9fc7a36a289 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:11:52.375183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7356fa9-c77d-46cc-b446-e9fc7a36a289 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:11:52.375456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7356fa9-c77d-46cc-b446-e9fc7a36a289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:11:52.385358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7356fa9-c77d-46cc-b446-e9fc7a36a289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:11:52.386898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7356fa9-c77d-46cc-b446-e9fc7a36a289 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:11:52.400686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 765/1539] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:12:02.028758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=11 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:12:02.392134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:15.113126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8620cba-1a22-42d7-ab6d-7dd35c96d264 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.124704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8620cba-1a22-42d7-ab6d-7dd35c96d264 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.124704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8620cba-1a22-42d7-ab6d-7dd35c96d264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.148246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d8620cba-1a22-42d7-ab6d-7dd35c96d264 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.149588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8620cba-1a22-42d7-ab6d-7dd35c96d264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.163999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 775/1540] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:15.171559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320492b-f25b-498a-b321-92ab25514750 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.172167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320492b-f25b-498a-b321-92ab25514750 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.172167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320492b-f25b-498a-b321-92ab25514750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.175070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.175351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.175621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.200081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6320492b-f25b-498a-b321-92ab25514750 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.201148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6320492b-f25b-498a-b321-92ab25514750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.207639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.209715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.214312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 766/1541] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.223923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 776/1542] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:15.230035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.230255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.230365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.230617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.244728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efb2e2-feea-4f97-a710-c3713f280578 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.257111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.257731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.258003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.258269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.258943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:15.259239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.261393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.267698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.268912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.281576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 777/1543] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:15.290485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 767/1544] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.291243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.296538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.309673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e18ddf-4a59-4836-a178-45cdc172c46c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.313151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27cc3a-53ce-407e-8c42-6fee2de5d7e9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.324498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.324809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.325116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.325392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.325519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.325716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.325825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.325922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.326295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:15.326538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.326681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.327577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:13:15.327577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:13:15.327577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:13:15.327869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:13:15.327869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:13:15.327869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:13:15.328464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd9670-2267-4599-b52d-88396e6587fc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.328755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ec714-bbc4-467d-984d-fdcba03724da admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.354042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 768/1545] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:13:15.367776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643d244-8988-4517-81bf-7a54765d3a00 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.368000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643d244-8988-4517-81bf-7a54765d3a00 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.368272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643d244-8988-4517-81bf-7a54765d3a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.400998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1643d244-8988-4517-81bf-7a54765d3a00 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.402609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1643d244-8988-4517-81bf-7a54765d3a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.415205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 769/1546] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.416646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.416945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.417184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.445016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.446611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.465129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 770/1547] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.465129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.468109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 778/1548] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:13:15.470179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3f3c-860b-491c-baa4-abc358511e57 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.471901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3f3c-860b-491c-baa4-abc358511e57 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.472429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3f3c-860b-491c-baa4-abc358511e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.475745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b1eea-dd27-4a1e-828d-88b0e0e3a2ae None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.492070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-caca3f3c-860b-491c-baa4-abc358511e57 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.493278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-caca3f3c-860b-491c-baa4-abc358511e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.532785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce196744-964d-400f-bf92-a2ed540b5d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.533443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196744-964d-400f-bf92-a2ed540b5d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.533722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196744-964d-400f-bf92-a2ed540b5d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.533996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196744-964d-400f-bf92-a2ed540b5d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.534644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196744-964d-400f-bf92-a2ed540b5d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.536577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196744-964d-400f-bf92-a2ed540b5d53 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.552288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 779/1549] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.553700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd7175-b3e2-44ff-b07f-4441df47fe62 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.553989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd7175-b3e2-44ff-b07f-4441df47fe62 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.554262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd7175-b3e2-44ff-b07f-4441df47fe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.574590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b5fd7175-b3e2-44ff-b07f-4441df47fe62 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.575936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5fd7175-b3e2-44ff-b07f-4441df47fe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.583937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 780/1550] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.585133 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eecde-fa5e-4329-8633-ff4e343387e8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.585415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eecde-fa5e-4329-8633-ff4e343387e8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.585688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eecde-fa5e-4329-8633-ff4e343387e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.605430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-193eecde-fa5e-4329-8633-ff4e343387e8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.606697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-193eecde-fa5e-4329-8633-ff4e343387e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.614622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 781/1551] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.615773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.616056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.616326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.635341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.636772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.644576 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 782/1552] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.645310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.655150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f597e-0a10-426c-ae8f-11de78f46795 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.659921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52abbf-260e-4992-b2bc-fecd12e20d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.660510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52abbf-260e-4992-b2bc-fecd12e20d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.660789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52abbf-260e-4992-b2bc-fecd12e20d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.661056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52abbf-260e-4992-b2bc-fecd12e20d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.661667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52abbf-260e-4992-b2bc-fecd12e20d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.661918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 771/1553] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:15.663317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8c6e0-3921-4cdd-88c9-e7ac2d7da50f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.663600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8c6e0-3921-4cdd-88c9-e7ac2d7da50f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.663738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52abbf-260e-4992-b2bc-fecd12e20d9b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.663919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8c6e0-3921-4cdd-88c9-e7ac2d7da50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.683353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e7e8c6e0-3921-4cdd-88c9-e7ac2d7da50f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.684541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e7e8c6e0-3921-4cdd-88c9-e7ac2d7da50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.740639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 783/1554] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:15.741807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.746238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 772/1555] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.746419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.746419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.746615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d679d2-cf4c-4b6b-bb4b-5d9227e68a21 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.746615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d679d2-cf4c-4b6b-bb4b-5d9227e68a21 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.746615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d679d2-cf4c-4b6b-bb4b-5d9227e68a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.763320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.770122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-49d679d2-cf4c-4b6b-bb4b-5d9227e68a21 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.770252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.770370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-49d679d2-cf4c-4b6b-bb4b-5d9227e68a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.772459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 784/1556] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.772772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 773/1557] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.777783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.777915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1525-867f-414c-8124-080f616c2d8f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.777915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1525-867f-414c-8124-080f616c2d8f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.777915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1525-867f-414c-8124-080f616c2d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.787872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3814e3-e100-459f-bf3b-9ebd1351f151 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.792908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.793524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.793801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.794068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.794756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:15.795060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.796679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.796818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-895a1525-867f-414c-8124-080f616c2d8f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.798044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-895a1525-867f-414c-8124-080f616c2d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.806845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 774/1558] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.808109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.808397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.808665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.809296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 785/1559] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:15.810003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.826061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7e78-b952-4cd2-8e60-3bdf14eb6f13 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.831178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.831769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.832047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.832326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.832775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.832933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.833960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.834882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.841865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 775/1560] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.846107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.852484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63c5fd-9059-446e-bc07-530808d0451b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.857207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.864068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.864068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.864068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.864068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.864068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.871199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 776/1561] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:15.872004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.881409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6251b-e784-4c2c-acd4-ca0b87ded593 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.886621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.887201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.887479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.887751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.888367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:15.888671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.890221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.901937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 777/1562] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:15.902867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.909054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:15.912521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fcd0a-407b-4cc5-81eb-00fe4e880100 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.917510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.918075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.918351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.918621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.919377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:15.919710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.921291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1180-5b71-467a-9811-be18e0552f7a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.933347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 778/1563] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:15.938556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.938556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.938556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.938865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 786/1564] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:15.938865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.953464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28884b-1837-4f94-807b-1f67e5a7f801 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.955821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:15.957187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:15.958562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.959219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.959500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.959763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.960390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.962350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.965363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 779/1565] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:15.969572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:15.976489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0266f6a-e274-4a3c-8f98-b8faff67d0d0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:15.981401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:15.982013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:15.982289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:15.982553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:15.983220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:15.983552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:15.985268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:15.998000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 780/1566] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:15.998752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.008149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22375d0e-eb02-4ec5-b913-d5322b6850ef admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.013480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.014053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.014337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.014605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.019049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:16.019049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.019049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.028696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 781/1567] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:16.029460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.039876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371d4aa-3a3a-4ec9-acc8-4e92ab810463 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.045435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 787/1568] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:15 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.045567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.046070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.046267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.046451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.046815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.047549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.049133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.057740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:16.060129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 782/1569] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:16.061226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.072277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bb37f-4d1c-4a90-b0a7-c41b1e011f27 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.114816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3bce0-aea0-4032-902b-3c6edc8844af admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.120736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.121333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.121608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.121876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.122613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.128146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.128514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.129112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.129387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.129657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.130291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.131927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.144707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 783/1570] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:16.144707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.152045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54002158-04fa-44e1-8494-f3bf88f53428 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.156508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.157064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.157338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.157607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.158205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.160103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.308295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 784/1571] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.309181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.345508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 788/1572] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/projects => generated 403 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.345508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.374306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926583f-23fc-40d4-a5ed-9304c9395bd3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.398797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:16.406031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d1d18-3b99-4a15-8dbc-e9bf08ae8b90 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.410526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.411305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.411590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.411863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.412486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.414400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.438266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.438896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.442360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.442360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.442360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:16.442360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.442360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.453238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 785/1573] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:16.453988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.464579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5e645-3464-41af-b9a0-1035c41c269f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.469188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.469747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.470018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.470285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.471160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.473021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.496141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 789/1574] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.496945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.558163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24da70-d8de-4846-afcc-455be8b72516 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.559283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 786/1575] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.560025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.580963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:16.617922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.618535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.618843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.619174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.619868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.621842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.623010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b8e1-1d76-42e2-b0d0-142dc0a5aa1a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.627586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.628224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.628495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.628790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] PATH_INFO: `/v3/projects/830b226057d740ca85bbdde6c1a5dca8/users/45dd5b35a039468d99ec0d393056c5af/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.643746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830b226057d740ca85bbdde6c1a5dca8, user_id=45dd5b35a039468d99ec0d393056c5af, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.646959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.729744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 790/1576] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.730568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.740261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 787/1577] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:16 2022] PUT /identity/v3/projects/830b226057d740ca85bbdde6c1a5dca8/users/45dd5b35a039468d99ec0d393056c5af/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:16.740420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7642b4-13dc-4b00-b8ac-12cd79d17ecc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.741130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.805456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.806151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.806424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.806716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.807389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.807689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335e132-cdf8-491e-83bd-a1b08286fd16 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.809013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.816842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.817376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.817677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.817947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.818904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.819639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:13:16.819639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:13:16.819639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:13:16.822245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.822377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 791/1578] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:16.822377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.834759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01036fd8-264e-46bd-97c5-de5451482eb3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.840021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.840819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.841102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.841392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.842005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.843951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.917731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 788/1579] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.918642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.928498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cb4e2-2fc5-4e74-a111-a3d241b118d1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.930232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 792/1580] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/users => generated 302 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:16.931198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:16.940930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44064e54-f222-45ab-8bc1-dad1aa484970 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:16.944110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.944859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.945133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.945401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.945995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:16.946141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.946620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:16.946930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:16.947207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:16.948318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:16.948610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.950304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:16.958831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:17.037780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 793/1581] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:17.038586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.048261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631e3f5-c1cc-475c-adb1-9609e577cd0a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.052893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e396e8-1e52-4916-ba61-831bbcaf5d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.053600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e396e8-1e52-4916-ba61-831bbcaf5d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.053886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e396e8-1e52-4916-ba61-831bbcaf5d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.054160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e396e8-1e52-4916-ba61-831bbcaf5d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.054845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e396e8-1e52-4916-ba61-831bbcaf5d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.056756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e396e8-1e52-4916-ba61-831bbcaf5d80 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.065446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 789/1582] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:16 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:17.065446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.124134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d64e18-be86-4e2e-a0a6-52053ca2feb3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.128851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.129458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.129736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.130041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.130692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.132295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.136926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 794/1583] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:17 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:17.136926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70991-b807-4960-870c-0519575af09a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.141577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70991-b807-4960-870c-0519575af09a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.141577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70991-b807-4960-870c-0519575af09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.145464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 790/1584] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.146172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.156805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d3760-60b5-4f30-bbdc-49af9d313dc4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.158446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f2c70991-b807-4960-870c-0519575af09a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:17.159712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2c70991-b807-4960-870c-0519575af09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:17.161844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.162595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.162908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.163183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] PATH_INFO: `/v3/projects/6da31a6c1eb84fa1aaf256037c8848e0/users/22e92975af1d439f88e33cc494c88b1f/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.167461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 795/1585] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:17.171068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70991-b807-4960-870c-0519575af09a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.179001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70991-b807-4960-870c-0519575af09a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.180127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da31a6c1eb84fa1aaf256037c8848e0, user_id=22e92975af1d439f88e33cc494c88b1f, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.183334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.184332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.184930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.185227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.185482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.186121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.187733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.198479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 796/1586] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.206645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.210253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8321180-04ab-458f-afd1-4904f9d6ba73 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.215234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.215800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.216076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.216346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.216975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.218530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.228385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 797/1587] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.229148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.238291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e080f5-b099-4b74-a508-fae0e5cfec97 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.242874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.243426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.243736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.243968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.245140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.246687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.256614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 798/1588] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.257347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.267594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48078bde-f04c-4b9d-ba14-aa8a86aa46a8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.272388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 791/1589] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:17 2022] PUT /identity/v3/projects/6da31a6c1eb84fa1aaf256037c8848e0/users/22e92975af1d439f88e33cc494c88b1f/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:17.281262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.281974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:17.329179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.329790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.330068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.330363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.331020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.332930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.337396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545aa29-f20d-4d1d-a5fa-80d3319769d6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.342097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.342772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.343050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.343319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.344081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.346039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.356056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 792/1590] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.356836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.366141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0a049-0e6a-4a2f-8b67-865fc0dcf872 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.371234 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.371790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.372067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.372334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.373005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.374602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.384226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 793/1591] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.384226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.420824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 799/1592] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:17 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:17.421743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.435594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5fe80-af88-471f-95c3-7c45c62b5f38 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.435800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:17.440298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.440738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db192a2-a79a-4df2-9a2b-910992ef0bd8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.440897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.441177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.441449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.442089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.443712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.444627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.445307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.445978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.445978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.454265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 800/1593] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.455195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.458475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.460063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.464690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cadd9-4642-4815-b74b-91a8b9164cce admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.469278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118df11-7792-4b62-bef7-f209c6daa22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.469584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 794/1594] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:17.469937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118df11-7792-4b62-bef7-f209c6daa22e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.470224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118df11-7792-4b62-bef7-f209c6daa22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.470347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.470504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118df11-7792-4b62-bef7-f209c6daa22e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.471477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118df11-7792-4b62-bef7-f209c6daa22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.473048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118df11-7792-4b62-bef7-f209c6daa22e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.486673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7aa64-0dbe-4f94-bb8e-acf57cafdd03 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.486673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.486673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.487043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 801/1595] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.487140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.487140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.487140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.488566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.488981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.489272 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.489545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.502604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 795/1596] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.502604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.508954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:17.510214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:17.513082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ee338-d607-4e9f-8521-68287fe9b39e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.518007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.518589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.518876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 802/1597] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:17.518985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.519160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.519808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.519923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.521517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.531390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458f053-4265-4245-9187-bc4a2b7aa425 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.536144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 796/1598] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.536144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.539602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.540277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.540554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.540840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] PATH_INFO: `/v3/projects/8e5a6e09a2be4042b0010a196f744030/users/5c252fc819d6480ea720098d14d02574/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.544658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed9912-a8a8-485c-814e-a351e7f9a365 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.549583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.549959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.550239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.550502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.551167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.552735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.556805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5a6e09a2be4042b0010a196f744030, user_id=5c252fc819d6480ea720098d14d02574, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.558426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:13:17.558426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:13:17.558426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:13:17.560419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.563325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 797/1599] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:17.564093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.573484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b03ce-a7eb-4fcd-b1e3-64382a997d5c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.578565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.584046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.584046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.584046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] PATH_INFO: `/v3/projects/3e942fa845c5449f8ae6cd08cbc008fb/users/8896ffa115fd4da69566177a7c8cadd2/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.593786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e942fa845c5449f8ae6cd08cbc008fb, user_id=8896ffa115fd4da69566177a7c8cadd2, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.596704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.683733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 803/1600] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:17 2022] PUT /identity/v3/projects/8e5a6e09a2be4042b0010a196f744030/users/5c252fc819d6480ea720098d14d02574/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:17.684614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.706092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 798/1601] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:17 2022] PUT /identity/v3/projects/3e942fa845c5449f8ae6cd08cbc008fb/users/8896ffa115fd4da69566177a7c8cadd2/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:17.706970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.756451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92d1f4-0683-4081-8ae8-dfc289b8433a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.768190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6476e-15cf-4698-972e-c9b1ec8dace8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.773240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.773905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.774181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.774455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] PATH_INFO: `/v3/projects/b0b3e39661374044a2aac6dd8a912af7/users/a24422d1f1e54ca5beff6b929b69a062/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.791919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b3e39661374044a2aac6dd8a912af7, user_id=a24422d1f1e54ca5beff6b929b69a062, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.805067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.834163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.834861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.835139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.835409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] PATH_INFO: `/v3/projects/bad151affd064e19b491eafed5d9d2e0/users/b4531fdcf853420196025230e44d6752/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.838527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:17.852678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad151affd064e19b491eafed5d9d2e0, user_id=b4531fdcf853420196025230e44d6752, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:17.856191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:17.889125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 799/1602] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:17 2022] PUT /identity/v3/projects/b0b3e39661374044a2aac6dd8a912af7/users/a24422d1f1e54ca5beff6b929b69a062/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:17.890958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:17.948649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89405596-5dbf-4fdd-8d46-2b7b2659f340 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:17.952379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:17.953166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:17.953445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:17.953715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:17.967549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 804/1603] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:17 2022] PUT /identity/v3/projects/bad151affd064e19b491eafed5d9d2e0/users/b4531fdcf853420196025230e44d6752/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:17.972160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.000137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:18.035516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.036198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab196-613b-43fc-8ebe-3e717f9e46c2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.039048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.042807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b172902-534d-4ea7-af10-f0e5b3167e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.043721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b172902-534d-4ea7-af10-f0e5b3167e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.044387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b172902-534d-4ea7-af10-f0e5b3167e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.044914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b172902-534d-4ea7-af10-f0e5b3167e3e admin admin] PATH_INFO: `/v3/projects/be0d52e189904e3c9971960d3ff4f58b/users/ffb46379ef204df6bebd58f3bdb78ff6/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.054221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 800/1604] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:18.054221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.067372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f1feb-94ef-4a9d-9d74-8a3946312a1d service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.068328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b172902-534d-4ea7-af10-f0e5b3167e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0d52e189904e3c9971960d3ff4f58b, user_id=ffb46379ef204df6bebd58f3bdb78ff6, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.073049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b172902-534d-4ea7-af10-f0e5b3167e3e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.074708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b803d7-3f5f-438e-9ec8-1c7a95162930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.075459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b803d7-3f5f-438e-9ec8-1c7a95162930 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.075801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b803d7-3f5f-438e-9ec8-1c7a95162930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.076133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b803d7-3f5f-438e-9ec8-1c7a95162930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.076992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b803d7-3f5f-438e-9ec8-1c7a95162930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.078988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b803d7-3f5f-438e-9ec8-1c7a95162930 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.095227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 801/1605] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:18.096798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da343-b27e-4d6f-b529-4fc88a7d9627 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.097081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da343-b27e-4d6f-b529-4fc88a7d9627 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.097372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da343-b27e-4d6f-b529-4fc88a7d9627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.125935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-030da343-b27e-4d6f-b529-4fc88a7d9627 None None] MFA Rules not processed for user `5c252fc819d6480ea720098d14d02574`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:18.127602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-030da343-b27e-4d6f-b529-4fc88a7d9627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.169413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 805/1606] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:17 2022] PUT /identity/v3/projects/be0d52e189904e3c9971960d3ff4f58b/users/ffb46379ef204df6bebd58f3bdb78ff6/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:18.170540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ceaae-3148-4acb-89bc-952e755ead3c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.171032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ceaae-3148-4acb-89bc-952e755ead3c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.171301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ceaae-3148-4acb-89bc-952e755ead3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.201679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 802/1607] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:18.203125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9384-546c-407c-bada-589de6397287 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.203479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9384-546c-407c-bada-589de6397287 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.204324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9384-546c-407c-bada-589de6397287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.210619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-556ceaae-3148-4acb-89bc-952e755ead3c None None] MFA Rules not processed for user `8896ffa115fd4da69566177a7c8cadd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:18.212070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-556ceaae-3148-4acb-89bc-952e755ead3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.240299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-184f9384-546c-407c-bada-589de6397287 None None] MFA Rules not processed for user `a24422d1f1e54ca5beff6b929b69a062`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:18.241973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-184f9384-546c-407c-bada-589de6397287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.303784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 806/1608] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:18.305814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9045d-5435-4c46-b562-5270fa41f7bc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.306908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9045d-5435-4c46-b562-5270fa41f7bc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.306908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9045d-5435-4c46-b562-5270fa41f7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.329882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 803/1609] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:18.331594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9384-546c-407c-bada-589de6397287 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.337418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-59e9045d-5435-4c46-b562-5270fa41f7bc None None] MFA Rules not processed for user `b4531fdcf853420196025230e44d6752`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:18.343294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-59e9045d-5435-4c46-b562-5270fa41f7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.374543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:18.394637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:18.408321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9384-546c-407c-bada-589de6397287 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.413959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.414984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.415368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.415710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] PATH_INFO: `/v3/projects/78a0df50573341beb4cf1e2480ba6736/users/c5f2e6ec64db4ba3b3db2ca564c26bed/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.437921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 807/1610] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:18.437921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f249a1-587a-4296-a810-bc1363fbdad9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.437921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f249a1-587a-4296-a810-bc1363fbdad9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.437921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f249a1-587a-4296-a810-bc1363fbdad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.439510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a0df50573341beb4cf1e2480ba6736, user_id=c5f2e6ec64db4ba3b3db2ca564c26bed, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.443241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.461035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-31f249a1-587a-4296-a810-bc1363fbdad9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:18.462351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-31f249a1-587a-4296-a810-bc1363fbdad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.471918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 808/1611] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:18.473350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180ba6-7124-4d35-aa15-3a98c87d1dbc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.473739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180ba6-7124-4d35-aa15-3a98c87d1dbc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.474117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180ba6-7124-4d35-aa15-3a98c87d1dbc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.475554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 809/1612] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 10 23:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:18.476615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180ba6-7124-4d35-aa15-3a98c87d1dbc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.491467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180ba6-7124-4d35-aa15-3a98c87d1dbc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.499861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.500985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.501488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.501862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.502617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.504963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.567276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 804/1613] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:18 2022] PUT /identity/v3/projects/78a0df50573341beb4cf1e2480ba6736/users/c5f2e6ec64db4ba3b3db2ca564c26bed/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:18.567276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.597633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 810/1614] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:18.597633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.633456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947221a-23d6-482b-8df8-3ff2b4cf7680 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.658295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e663d0c-4d88-4076-8a83-ee259d0fb5b3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.662982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619de8-3bab-42cb-978c-5de0e9e79c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.663611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619de8-3bab-42cb-978c-5de0e9e79c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.663887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619de8-3bab-42cb-978c-5de0e9e79c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.664156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619de8-3bab-42cb-978c-5de0e9e79c31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.664786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619de8-3bab-42cb-978c-5de0e9e79c31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.666703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619de8-3bab-42cb-978c-5de0e9e79c31 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.691092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.691688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.691965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.692230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.692842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:18.694790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:18.799476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 811/1615] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/projects => generated 403 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:18.803246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52257931-6b53-4054-813e-688d6d51ea39 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.803246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52257931-6b53-4054-813e-688d6d51ea39 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.803246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52257931-6b53-4054-813e-688d6d51ea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.837040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 805/1616] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:18.838268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:18.840270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.867622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-52257931-6b53-4054-813e-688d6d51ea39 None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:18.869146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52257931-6b53-4054-813e-688d6d51ea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:18.900499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:18.905589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e3974-4b2a-4ffb-862a-81cb276895f6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:18.909942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:18.910619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.910923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.911272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:18.973236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 812/1617] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:18.973236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:18.973236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:18.973236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.006747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:19.013331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:19.019787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.021176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.032044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 813/1618] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:19.032044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.040097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 806/1619] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:19.040924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.050846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e0f71-07a4-498b-9b8c-ac6f1c10f3fa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.060152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d82b8-c382-4f4c-9134-5fc43dd0c0a0 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.060733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.061427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.061686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.061957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.062632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.064724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.066977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.068029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.068552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.068978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.069789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.072116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.179337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:19.189111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 807/1620] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:19 2022] POST /identity/v3/users => generated 308 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:19.190142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.190683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 814/1621] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:19 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:19.191579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.204166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143cc6f-d5b5-471d-9dad-4e63e6cc30b6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.206376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710fbf5-f05e-4815-b5c3-5f998b6c1d11 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.210551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.211811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.212252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.212851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.214345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.214478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.215279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.215888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.215888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.217963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.289597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.291183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.300894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 815/1622] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:19.301749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.320440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e9af1-67a9-44a1-b11f-3a685bc354ea service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.325019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.325701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.326012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.326311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.339267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.340710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.353234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 816/1623] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:19.354095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.364776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd613-e963-4be1-b7d1-38e9fbd89508 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.371604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.372225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.373044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.373044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.373504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.375427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.387748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 808/1624] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:19 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:19.388874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.409248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 817/1625] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:19.409248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.412320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484653ea-753c-4259-b8b5-b577624e8221 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.422104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.423706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.423706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.423931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.427580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.433701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.434438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2a6c-31a9-4113-801c-783e7d5b9e71 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.445379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.446490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.448285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.451367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.451985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.454569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.464272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 809/1626] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:19.464867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.479517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 818/1627] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:19.481464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.487755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744f2ec-65e3-4beb-ad35-4a61753dcf49 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.499797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad2fb-9959-4b32-8a2f-271e1e4c0974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.500490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad2fb-9959-4b32-8a2f-271e1e4c0974 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.501913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad2fb-9959-4b32-8a2f-271e1e4c0974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.502201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad2fb-9959-4b32-8a2f-271e1e4c0974 admin admin] PATH_INFO: `/v3/projects/ee8eab2601844530bd3cacdc68d249ad/users/5f416f2799bf44b5a61b52cab4a77318/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.505861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15910-fd68-4ff9-a3e1-53b91e2bff3c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.518508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.520052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.520897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.520897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] PATH_INFO: `/v3/projects/bed6379f7d0c4d178f3388879540f716/users/a127941b76ad475bad8b5afd5dc91864/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.539583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad2fb-9959-4b32-8a2f-271e1e4c0974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8eab2601844530bd3cacdc68d249ad, user_id=5f416f2799bf44b5a61b52cab4a77318, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.546570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad2fb-9959-4b32-8a2f-271e1e4c0974 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.562020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed6379f7d0c4d178f3388879540f716, user_id=a127941b76ad475bad8b5afd5dc91864, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.565128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.665863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 819/1629] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:19 2022] PUT /identity/v3/projects/bed6379f7d0c4d178f3388879540f716/users/a127941b76ad475bad8b5afd5dc91864/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:19.666148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 810/1629] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:19 2022] PUT /identity/v3/projects/ee8eab2601844530bd3cacdc68d249ad/users/5f416f2799bf44b5a61b52cab4a77318/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:19.667320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.668599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d1473-af46-46fc-8a72-663670fb551a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.669022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d1473-af46-46fc-8a72-663670fb551a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.669782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d1473-af46-46fc-8a72-663670fb551a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.671477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 811/1630] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 10 23:13:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:19.676532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d1473-af46-46fc-8a72-663670fb551a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.785940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224f3d5-7848-46ae-8245-42fecd7c2838 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.795511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.796109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.796389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.796665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] PATH_INFO: `/v3/projects/b681e409e9d4473ca2db0f72748c9930/users/695bb45a8051489a910f75acb505ac26/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.801890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d1473-af46-46fc-8a72-663670fb551a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.809163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.809928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.810210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.811562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.812283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.815402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.830142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 812/1631] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:19.831028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.841884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b681e409e9d4473ca2db0f72748c9930, user_id=695bb45a8051489a910f75acb505ac26, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.852411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.859159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bdc65-88ed-4058-93e6-cae1137db1a7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:19.866039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59184b-a4b3-4407-9736-b2a67f4038b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:19.866752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59184b-a4b3-4407-9736-b2a67f4038b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.867036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59184b-a4b3-4407-9736-b2a67f4038b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.867805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59184b-a4b3-4407-9736-b2a67f4038b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.868060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59184b-a4b3-4407-9736-b2a67f4038b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:19.870418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59184b-a4b3-4407-9736-b2a67f4038b9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:19.890262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 813/1632] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:19.892193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:19.892479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:19.892930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:19.922547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:19.924444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:19.937229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 820/1633] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:19 2022] PUT /identity/v3/projects/b681e409e9d4473ca2db0f72748c9930/users/695bb45a8051489a910f75acb505ac26/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:19.938243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:19.946849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:19.971509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:20.028789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835f03f-2a17-4817-b7df-6562d44eee76 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:20.045356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:20.045356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:20.045356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:20.045356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] PATH_INFO: `/v3/projects/ee8eab2601844530bd3cacdc68d249ad/users/5f416f2799bf44b5a61b52cab4a77318/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:20.045900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8eab2601844530bd3cacdc68d249ad, user_id=5f416f2799bf44b5a61b52cab4a77318, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:20.051487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:20.072144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 821/1634] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:19 2022] PUT /identity/v3/projects/ee8eab2601844530bd3cacdc68d249ad/users/5f416f2799bf44b5a61b52cab4a77318/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:20.079837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:20.093486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 814/1635] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:13:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:20.094491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:20.113087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23516967-c358-4469-b473-7fdfc111ea9f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:20.215577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d0f88-53b7-460f-926d-8d81d58ae74a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:20.220906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:20.221513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:20.221790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:20.222059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:20.222362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e372a47-007a-4c30-9973-9798b5549370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:20.222719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:20.223117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e372a47-007a-4c30-9973-9798b5549370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:20.223401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e372a47-007a-4c30-9973-9798b5549370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:20.223673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e372a47-007a-4c30-9973-9798b5549370 admin admin] PATH_INFO: `/v3/projects/bed6379f7d0c4d178f3388879540f716/users/a127941b76ad475bad8b5afd5dc91864/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:20.225865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:20.226609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e372a47-007a-4c30-9973-9798b5549370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed6379f7d0c4d178f3388879540f716, user_id=a127941b76ad475bad8b5afd5dc91864, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:20.233270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e372a47-007a-4c30-9973-9798b5549370 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:20.249364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 822/1636] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:20 2022] PUT /identity/v3/projects/bed6379f7d0c4d178f3388879540f716/users/a127941b76ad475bad8b5afd5dc91864/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:20.251620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda1b87-18f4-4ca3-bb6f-8523fe43e33e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:20.251758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda1b87-18f4-4ca3-bb6f-8523fe43e33e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:20.251994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda1b87-18f4-4ca3-bb6f-8523fe43e33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:20.256118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 815/1637] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:20.257193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:20.298455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6bda1b87-18f4-4ca3-bb6f-8523fe43e33e None None] MFA Rules not processed for user `5f416f2799bf44b5a61b52cab4a77318`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:20.300136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6bda1b87-18f4-4ca3-bb6f-8523fe43e33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:20.465588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693396b4-066f-40b3-a027-b06cf01021a8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:20.497014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 823/1638] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:20.498703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:20.500048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:20.500886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:20.539973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:20.540753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:20.541036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:20.541943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:20.575311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] MFA Rules not processed for user `a127941b76ad475bad8b5afd5dc91864`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:20.576549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:20.685528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:20.688191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:20.703036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 816/1639] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:20.704361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:20.737616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 824/1640] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:20 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:20.738930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:20.750035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c81d13-5049-4383-a09e-ebb896d81607 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:21.013163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:21.013163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:21.013163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:21.013163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:21.019560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02db63a-2a3c-4c81-80fc-e8bbc2360585 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:21.019560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324719-b22e-4122-911c-db3843e5d659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:21.019560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324719-b22e-4122-911c-db3843e5d659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:21.019560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324719-b22e-4122-911c-db3843e5d659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:21.020721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324719-b22e-4122-911c-db3843e5d659 admin admin] PATH_INFO: `/v3/projects/b681e409e9d4473ca2db0f72748c9930/users/695bb45a8051489a910f75acb505ac26/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:21.020721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324719-b22e-4122-911c-db3843e5d659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b681e409e9d4473ca2db0f72748c9930, user_id=695bb45a8051489a910f75acb505ac26, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:21.020721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324719-b22e-4122-911c-db3843e5d659 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:21.020721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 817/1641] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:20 2022] PUT /identity/v3/projects/b681e409e9d4473ca2db0f72748c9930/users/695bb45a8051489a910f75acb505ac26/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:21.021805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:21.021805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:21.022734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:21.022734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:21.022734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:21.023436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 825/1642] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:21.024211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] MFA Rules not processed for user `695bb45a8051489a910f75acb505ac26`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:21.024211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:21.156677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 818/1643] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:21.605289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:21.706386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:22.138462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:22.150450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea80b1-4015-44ba-8000-0e073e134d46 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:22.255325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:22.256051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:22.256336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:22.256713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:22.361177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:22.362926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:22.377541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 826/1644] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:22 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:13:22.398685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:22.420672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab21800-c375-45a8-bc7a-4381bfebc48e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:22.435380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8243531-3413-42ad-a6ad-2cf19b65ecad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:22.441442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8243531-3413-42ad-a6ad-2cf19b65ecad service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:22.441442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8243531-3413-42ad-a6ad-2cf19b65ecad service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:22.441442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8243531-3413-42ad-a6ad-2cf19b65ecad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:22.456595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8243531-3413-42ad-a6ad-2cf19b65ecad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:22.458086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8243531-3413-42ad-a6ad-2cf19b65ecad service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:22.482641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 819/1645] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:22.812390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:22.964461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179ef4-d011-4b41-a113-ee52a73f7d8b service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:22.980181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:22.981076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:22.981076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:22.981522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] PATH_INFO: `/v3/users/b4531fdcf853420196025230e44d6752` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:22.984979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4531fdcf853420196025230e44d6752)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:22.984979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:13:22.984979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 10 23:13:22.984979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: Aug 10 23:13:22.987363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:23.040235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20482a0d-1069-4106-8b61-eae402d21322 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.041294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20482a0d-1069-4106-8b61-eae402d21322 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.041725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20482a0d-1069-4106-8b61-eae402d21322 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.044105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 820/1646] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 10 23:13:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:23.078002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546966ab-65f7-4693-af9e-c0ca17d8980e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.078289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546966ab-65f7-4693-af9e-c0ca17d8980e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.078564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546966ab-65f7-4693-af9e-c0ca17d8980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.098865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:23.130088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-546966ab-65f7-4693-af9e-c0ca17d8980e None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:23.131816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-546966ab-65f7-4693-af9e-c0ca17d8980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:23.285141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 821/1647] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 10 23:13:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 10 23:13:23.286958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb552c-3249-4934-a16a-746c6394e49f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.287178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb552c-3249-4934-a16a-746c6394e49f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.287604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb552c-3249-4934-a16a-746c6394e49f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.296178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 822/1648] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 10 23:13:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:23.297736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.298151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.298526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.326291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:23.327959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:23.346557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 823/1649] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:13:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:23.347567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:23.357244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4531fdcf853420196025230e44d6752'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:23.362168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8342580-99ea-48f4-a380-27467d3bbd81 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:23.366593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:23.367628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.368109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.368287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.405041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4531fdcf853420196025230e44d6752'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:23.432397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4531fdcf853420196025230e44d6752'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:23.435631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4531fdcf853420196025230e44d6752'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:23.437614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4531fdcf853420196025230e44d6752'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:23.489079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:23.499979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:23.502787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:23.515286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 824/1650] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:23.516352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:23.527412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31c9d1-e9f2-4f93-ad3b-9d86a5b45605 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:23.531746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:23.532638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.533273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.533687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.536110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:23.554296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:23.557934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16292dea-6b9a-4845-9b5b-2d9e7eaedce2 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:23.573404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 825/1651] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:23.585972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.586276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.586545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.592544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 827/1652] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:22 2022] DELETE /identity/v3/users/b4531fdcf853420196025230e44d6752 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:23.609661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:23.616018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:23.683104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 826/1653] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:23.787193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:23.788892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:23.908589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55280f7-48ee-4542-bea6-c598bd9c7480 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:23.913644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:23.914890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:23.915480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:23.915878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:23.928657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:23.931043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:23.966449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 828/1654] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:26.519670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:26.522199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:26.542830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029fcb6-2f41-4fb6-a3f3-1b52c49dbf67 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:26.554203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:26.564683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:26.565657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:26.568406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] PATH_INFO: `/v3/projects/bad151affd064e19b491eafed5d9d2e0` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:26.570112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad151affd064e19b491eafed5d9d2e0)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:26.571590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 23:13:26.571590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 10 23:13:26.571590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: Aug 10 23:13:26.576339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:26.590496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:26.799981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad151affd064e19b491eafed5d9d2e0', 'request_id': 'req-1d7faa84-1cdb-4337-a84a-987bd17101b3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:26.867009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad151affd064e19b491eafed5d9d2e0', 'request_id': 'req-1d7faa84-1cdb-4337-a84a-987bd17101b3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:26.876870 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad151affd064e19b491eafed5d9d2e0', 'request_id': 'req-1d7faa84-1cdb-4337-a84a-987bd17101b3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:26.940115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] The token cache is being invalidate because project bad151affd064e19b491eafed5d9d2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:26.940413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:26.941619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:26.945295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 827/1655] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:26 2022] DELETE /identity/v3/projects/bad151affd064e19b491eafed5d9d2e0 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:26.950114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:26.961459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1d3b-5adb-42c2-992c-50e4a34ca34b service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:27.033488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:27.034107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:27.047672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:27.141933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:27.142984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:27.143430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:27.143840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] PATH_INFO: `/v3/users/a127941b76ad475bad8b5afd5dc91864` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:27.150618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a127941b76ad475bad8b5afd5dc91864)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:27.152650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:27.209628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7faa84-1cdb-4337-a84a-987bd17101b3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:27.226763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:27.227421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:27.227697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:27.227962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] PATH_INFO: `/v3/users/5c252fc819d6480ea720098d14d02574` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:27.232546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c252fc819d6480ea720098d14d02574)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:27.234490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:27.417427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a127941b76ad475bad8b5afd5dc91864'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.446585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a127941b76ad475bad8b5afd5dc91864'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.457032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a127941b76ad475bad8b5afd5dc91864'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.457970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a127941b76ad475bad8b5afd5dc91864'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.458837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50d8063f-5a6e-4780-a64a-b1a5f6c7bf34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a127941b76ad475bad8b5afd5dc91864'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.559328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c252fc819d6480ea720098d14d02574'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.596130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c252fc819d6480ea720098d14d02574'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.602416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 829/1656] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:26 2022] DELETE /identity/v3/users/a127941b76ad475bad8b5afd5dc91864 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:27.614194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c252fc819d6480ea720098d14d02574'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.615842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c252fc819d6480ea720098d14d02574'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.628167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c252fc819d6480ea720098d14d02574'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:27.730484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 828/1657] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:27 2022] DELETE /identity/v3/users/5c252fc819d6480ea720098d14d02574 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:27.734898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:27.735405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:27.735898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:27.783790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:27.785308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:27.997358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 830/1658] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:28.017039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:28.019893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:28.135334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:28.296846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10aace-ab01-4eb7-876a-3cd914dcb71d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:28.307482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1167eb-fbcf-4fb0-9c66-623bd687120e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:28.308484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1167eb-fbcf-4fb0-9c66-623bd687120e service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:28.312936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1167eb-fbcf-4fb0-9c66-623bd687120e service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:28.312936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1167eb-fbcf-4fb0-9c66-623bd687120e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:28.340789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1167eb-fbcf-4fb0-9c66-623bd687120e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:28.342778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1167eb-fbcf-4fb0-9c66-623bd687120e service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:28.376325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 829/1659] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:29.431371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:29.433069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:29.454297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186b47-e8ca-49fb-8883-9e4ffc6ef19e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:29.460293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:29.466399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:29.466399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:29.466399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] PATH_INFO: `/v3/projects/8e5a6e09a2be4042b0010a196f744030` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:29.466399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e5a6e09a2be4042b0010a196f744030)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:29.468058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:29.597436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:29.602356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e5a6e09a2be4042b0010a196f744030', 'request_id': 'req-90221732-913f-4aa7-9369-4ada20a9003e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:29.675974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e5a6e09a2be4042b0010a196f744030', 'request_id': 'req-90221732-913f-4aa7-9369-4ada20a9003e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:29.686217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e5a6e09a2be4042b0010a196f744030', 'request_id': 'req-90221732-913f-4aa7-9369-4ada20a9003e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:29.736441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] The token cache is being invalidate because project 8e5a6e09a2be4042b0010a196f744030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:29.737262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:29.738496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-90221732-913f-4aa7-9369-4ada20a9003e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:29.743513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 831/1660] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:29 2022] DELETE /identity/v3/projects/8e5a6e09a2be4042b0010a196f744030 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:29.746233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:29.748029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:29.748978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:29.801249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:29.802491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:29.966996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 830/1661] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:29.973413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:29.973768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:29.974235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.014373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:30.015920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:30.035971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 832/1662] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:30.040149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.057181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb520e0-16fe-4116-bba2-534d4e5576eb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.075544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.076424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.076779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.077121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.077849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:30.078347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.081079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.113669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 831/1663] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:30.118533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.129944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ea668-30ad-4797-9e34-720e05ec6dac None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.137031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347800d-c880-4b59-b3d9-52f05019d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.147429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347800d-c880-4b59-b3d9-52f05019d4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.148201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347800d-c880-4b59-b3d9-52f05019d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.151738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347800d-c880-4b59-b3d9-52f05019d4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.152557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347800d-c880-4b59-b3d9-52f05019d4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.154587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347800d-c880-4b59-b3d9-52f05019d4ea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.312687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 833/1664] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:30 2022] POST /identity/v3/projects => generated 357 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:30.317277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.344893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d89cd8-767a-4397-be9c-1eb76eaa398d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.493470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.493470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.493470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.493470 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.493887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.493887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.496800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757f4a1-5d16-4c90-a788-74a391bb4219 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.497284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757f4a1-5d16-4c90-a788-74a391bb4219 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.497383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757f4a1-5d16-4c90-a788-74a391bb4219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.549323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c757f4a1-5d16-4c90-a788-74a391bb4219 None None] MFA Rules not processed for user `ffb46379ef204df6bebd58f3bdb78ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:30.552211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c757f4a1-5d16-4c90-a788-74a391bb4219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:30.619861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 832/1665] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:30 2022] POST /identity/v3/users => generated 279 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:30.620999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.640774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab9b9f-1c28-4d96-b9e6-fd5448d0abe6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.647460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.648202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.648595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.648976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.649892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.652804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.661346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 834/1666] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:30.675674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.676128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.676539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.678305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 833/1667] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:30.683950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.700874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fccc8-d62f-40dc-8a33-9824a3b5f672 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.707593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.708217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.708633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.708919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.709665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.711386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.731132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 834/1668] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:13:30.732145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.734959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] MFA Rules not processed for user `ffb46379ef204df6bebd58f3bdb78ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:30.736185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:30.750170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e30630-f232-4d8f-a2b8-fd29f73d9192 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.754214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.755648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.756115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.756533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] PATH_INFO: `/v3/projects/6757209a9817405894b5a5d960511114/users/0650de5d486641f1adc09b40f64b2fa0/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.760068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 835/1669] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:30.760983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.762265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:30.779187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:30.784289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6757209a9817405894b5a5d960511114, user_id=0650de5d486641f1adc09b40f64b2fa0, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.785906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbefbca-89a0-45d4-9edc-dca8ec96395a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:30.788671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.813171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:30.813837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:30.814256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:30.814701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] PATH_INFO: `/v3/projects/bed6379f7d0c4d178f3388879540f716` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:30.818999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed6379f7d0c4d178f3388879540f716)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:30.838826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:30.882849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 835/1670] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:30 2022] PUT /identity/v3/projects/6757209a9817405894b5a5d960511114/users/0650de5d486641f1adc09b40f64b2fa0/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:30.893678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:30.897287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:31.070420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed6379f7d0c4d178f3388879540f716', 'request_id': 'req-330462d7-d27f-4f2e-ac22-157907aba0d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:31.130160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de76bb-fbcb-44b5-8595-b98e0d096ca3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:31.173027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed6379f7d0c4d178f3388879540f716', 'request_id': 'req-330462d7-d27f-4f2e-ac22-157907aba0d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:31.192988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed6379f7d0c4d178f3388879540f716', 'request_id': 'req-330462d7-d27f-4f2e-ac22-157907aba0d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:31.215026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] The token cache is being invalidate because project bed6379f7d0c4d178f3388879540f716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:31.215551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:31.217281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-330462d7-d27f-4f2e-ac22-157907aba0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:31.219654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 836/1671] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:30 2022] DELETE /identity/v3/projects/bed6379f7d0c4d178f3388879540f716 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:31.221356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205356ba-38f3-4a3c-8f79-7d1d60bfd082 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.221768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205356ba-38f3-4a3c-8f79-7d1d60bfd082 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.222164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205356ba-38f3-4a3c-8f79-7d1d60bfd082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.253621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:31.258458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:31.259283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.259653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.260589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.263946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-205356ba-38f3-4a3c-8f79-7d1d60bfd082 None None] MFA Rules not processed for user `0650de5d486641f1adc09b40f64b2fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:31.265488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-205356ba-38f3-4a3c-8f79-7d1d60bfd082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:31.265781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:31.385689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 837/1672] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:31.387141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a2c0c-8f15-4456-8a4f-e382418d5f57 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.387452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a2c0c-8f15-4456-8a4f-e382418d5f57 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.387709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a2c0c-8f15-4456-8a4f-e382418d5f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.398900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:31.401155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:31.426768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b86a2c0c-8f15-4456-8a4f-e382418d5f57 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:31.430876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 836/1673] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:31.432390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:31.436113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b86a2c0c-8f15-4456-8a4f-e382418d5f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:31.437538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:31.541669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:31.593577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 838/1674] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:31.605180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.605180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.605180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.661233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:31.668045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:31.711214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 839/1675] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:31.713381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:31.725653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179095-b3d6-458a-be67-a16157bd7a19 service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:31.740572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac269e-cc99-47d9-bb06-ca0e761554d9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:31.741924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:31.742941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.745031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.745322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.745971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:31.750713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:31.762811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:31.763439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.766230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.767477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.768746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:31.771701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:31.774843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:31.825189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 840/1676] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:31.826122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:31.844822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89af5f-caca-4c91-a8a4-be95b3505a34 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:31.858442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:31.860374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:31.860654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:31.860925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:31.861715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:31.866436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:31.926031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 837/1677] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:31 2022] POST /identity/v3/projects => generated 357 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:31.929849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.012945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 841/1678] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:31 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:32.014204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.039614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f8769-9065-4aa0-8f0f-bb1e09478780 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.105522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822e196-6fce-472d-aec6-3d9b3ab6bd97 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.111092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdcd36-8259-4c43-9e40-184ce3a6edf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.111839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdcd36-8259-4c43-9e40-184ce3a6edf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.112334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdcd36-8259-4c43-9e40-184ce3a6edf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.112454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdcd36-8259-4c43-9e40-184ce3a6edf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.113069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdcd36-8259-4c43-9e40-184ce3a6edf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.115682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdcd36-8259-4c43-9e40-184ce3a6edf4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.163436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.164508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.164897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.165924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.168149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.171167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.259087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 842/1679] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:32 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:32.261789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.263136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.263789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.290210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 838/1680] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:31 2022] POST /identity/v3/users => generated 285 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:32.292981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.328506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d7a4-fc7e-43ed-a91c-e069e6c01488 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.340080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.340804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.341042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.341315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.341963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.345193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.377425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 839/1681] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:32.378417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.381580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] MFA Rules not processed for user `45dd5b35a039468d99ec0d393056c5af`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:32.385227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:32.404319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16558fb-e8d6-434d-b734-fe36f4e7c727 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.432626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.433264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.433543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.433810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.441620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.443411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.470510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 840/1682] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:32.475135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.506693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8261d0c-ea90-42b4-a97a-80670cd7c6d1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.514551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 843/1683] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:32.516197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.524566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:32.544988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.545658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.545995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.546399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.550254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.554138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.602966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 841/1684] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:32.602966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.629367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb383922-ad07-4253-85c7-77aa272a0421 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.652277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:32.654513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.655223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.656204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.656204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] PATH_INFO: `/v3/projects/09dd90d18a8348328bf224b3e649d8cf/users/a7ece44fb4a547328ff41b1dcac7662a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.714716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:32.714716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dd90d18a8348328bf224b3e649d8cf, user_id=a7ece44fb4a547328ff41b1dcac7662a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.715489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49599d57-ebfe-4b77-b5d4-cb4a95fe9e68 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:32.715572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.724246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:32.726622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:32.726949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:32.728419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:32.734880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:32.899502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:32.905812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:32.928227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 844/1685] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:32.930606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:32.935043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:32.953648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 842/1686] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:32 2022] PUT /identity/v3/projects/09dd90d18a8348328bf224b3e649d8cf/users/a7ece44fb4a547328ff41b1dcac7662a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:32.956079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:33.071388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972497aa-b874-4c32-b896-d6a14c085afd service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:33.222876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:33.223630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.224011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.229284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.233648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfac55-16c9-4300-9559-db69598d81f8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:33.240608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d564f-22e7-43d7-9bd4-49019b31617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:33.241522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d564f-22e7-43d7-9bd4-49019b31617f admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.241954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d564f-22e7-43d7-9bd4-49019b31617f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.242378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d564f-22e7-43d7-9bd4-49019b31617f admin admin] PATH_INFO: `/v3/projects/9dedd403a353497cba9656dcb56b1cfb/users/6d18d72df0754cddb04c40eca228227e/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.265523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d564f-22e7-43d7-9bd4-49019b31617f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dedd403a353497cba9656dcb56b1cfb, user_id=6d18d72df0754cddb04c40eca228227e, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:33.268484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d564f-22e7-43d7-9bd4-49019b31617f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:33.369758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:33.371556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:33.394137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 845/1687] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:33.402850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:33.436526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 843/1688] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:32 2022] PUT /identity/v3/projects/9dedd403a353497cba9656dcb56b1cfb/users/6d18d72df0754cddb04c40eca228227e/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:33.437682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce4b2f-9b77-4d73-86e5-00864ffbe802 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.438082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce4b2f-9b77-4d73-86e5-00864ffbe802 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.438357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce4b2f-9b77-4d73-86e5-00864ffbe802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.451786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec86372-cb23-455c-bd2c-1feda030a955 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:33.515198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-44ce4b2f-9b77-4d73-86e5-00864ffbe802 None None] MFA Rules not processed for user `22e92975af1d439f88e33cc494c88b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:33.517244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-44ce4b2f-9b77-4d73-86e5-00864ffbe802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:33.570541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:33.597612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32256c-a49d-4d10-8de5-50a68aff14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:33.598238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32256c-a49d-4d10-8de5-50a68aff14d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.598520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32256c-a49d-4d10-8de5-50a68aff14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.598842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32256c-a49d-4d10-8de5-50a68aff14d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.599517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32256c-a49d-4d10-8de5-50a68aff14d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:33.603333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32256c-a49d-4d10-8de5-50a68aff14d1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:33.623088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 844/1689] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:33.624819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31efff-0eb3-44d6-9bf9-c80a8c58e8ab None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.625582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31efff-0eb3-44d6-9bf9-c80a8c58e8ab None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.625881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31efff-0eb3-44d6-9bf9-c80a8c58e8ab None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.627468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 845/1690] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 10 23:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:33.630860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.633388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.633874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.639334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 846/1691] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:33.640700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.641067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.641234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.715624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] MFA Rules not processed for user `6d18d72df0754cddb04c40eca228227e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:33.716925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:33.761229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:33.762550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:33.830221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:33.851502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 846/1692] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:33.853931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:33.886249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 847/1693] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:13:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:33.888539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:33.904527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d40064-003e-488d-8576-663ecf7b1ee1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:33.921075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:33.921897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:33.922057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:33.922361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] PATH_INFO: `/v3/projects/09dd90d18a8348328bf224b3e649d8cf/users/a7ece44fb4a547328ff41b1dcac7662a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:33.928481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dd90d18a8348328bf224b3e649d8cf, user_id=a7ece44fb4a547328ff41b1dcac7662a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:33.933045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:33.958429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 847/1694] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:33 2022] PUT /identity/v3/projects/09dd90d18a8348328bf224b3e649d8cf/users/a7ece44fb4a547328ff41b1dcac7662a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:33.959301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:34.057285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431928d-b7e9-4c7b-addc-43a52ffa8920 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:34.105386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a14e0-9593-4e56-a430-3ec1fc164534 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:34.111303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:34.113414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:34.113934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:34.114389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:34.115249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:34.117804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:34.262853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0193-75a1-4715-8ff1-e4ce0c87b957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:34.263754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0193-75a1-4715-8ff1-e4ce0c87b957 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:34.264291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0193-75a1-4715-8ff1-e4ce0c87b957 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:34.265029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0193-75a1-4715-8ff1-e4ce0c87b957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:34.336843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 848/1695] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:33 2022] POST /identity/v3/projects => generated 371 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:34.345061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:34.353104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:34.397427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0193-75a1-4715-8ff1-e4ce0c87b957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:34.397427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0193-75a1-4715-8ff1-e4ce0c87b957 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:34.420988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 848/1696] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:34.422354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:34.422648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:34.422965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:34.490275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] MFA Rules not processed for user `a7ece44fb4a547328ff41b1dcac7662a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:34.491614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:34.543289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b16c5-eebe-40f7-9e77-9866af60cb65 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:34.556448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:34.557927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:34.558314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:34.558675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:34.571249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:34.577464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:34.585509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 849/1697] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:34.586865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:34.737097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 849/1698] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:34.752530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:34.782292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98805a-bf95-4423-93de-9dba9dd2859f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:34.815206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:34.815841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:34.816129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:34.816407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:34.817046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:34.830933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:34.857119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d496747-3207-41d3-8059-d3452a27dad7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:34.862212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:34.866572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:34.867097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:34.867579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:34.912868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:34.915686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:34.969096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 850/1699] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:13:34.978585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 850/1700] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:34 2022] POST /identity/v3/users => generated 292 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:34.979895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:35.026916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916ee9d-05fd-416c-b2a3-d708bdb5c83b service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:35.048680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:35.049121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.049401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.049673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.050332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:35.052131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:35.075907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 851/1701] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:35.077136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:35.091345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523704f-a571-450e-a303-e4c6d166df26 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:35.103117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:35.103830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.104113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.104391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] PATH_INFO: `/v3/projects/f59447b4b9b7468f9c6b11ddfd3e8bb1/users/19c7b58c350946b592acd469b17ca3df/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.142231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59447b4b9b7468f9c6b11ddfd3e8bb1, user_id=19c7b58c350946b592acd469b17ca3df, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:35.148905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:35.262266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 851/1702] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:35 2022] PUT /identity/v3/projects/f59447b4b9b7468f9c6b11ddfd3e8bb1/users/19c7b58c350946b592acd469b17ca3df/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:35.266374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:35.278637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03782a6d-350b-4687-8352-c9266b1e3d08 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:35.394887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:35.400381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:35.400381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.400381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.400381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.400834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:35.402519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:35.430344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 852/1703] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:35.439366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:35.451543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64642c61-970c-4443-ae4f-00e98d7a93ae admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:35.457192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7fd95-ece6-4b9e-abd8-6f8a45c7a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:35.457427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7fd95-ece6-4b9e-abd8-6f8a45c7a826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.457716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7fd95-ece6-4b9e-abd8-6f8a45c7a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.457990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7fd95-ece6-4b9e-abd8-6f8a45c7a826 admin admin] PATH_INFO: `/v3/projects/f59447b4b9b7468f9c6b11ddfd3e8bb1/users/19c7b58c350946b592acd469b17ca3df/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.462304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7fd95-ece6-4b9e-abd8-6f8a45c7a826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59447b4b9b7468f9c6b11ddfd3e8bb1, user_id=19c7b58c350946b592acd469b17ca3df, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:35.468284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7fd95-ece6-4b9e-abd8-6f8a45c7a826 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:35.493430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 853/1704] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:35 2022] PUT /identity/v3/projects/f59447b4b9b7468f9c6b11ddfd3e8bb1/users/19c7b58c350946b592acd469b17ca3df/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:35.506461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.506781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.507924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.548009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cedec-bbff-4c49-9305-872c545ed501 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:35.606573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] MFA Rules not processed for user `19c7b58c350946b592acd469b17ca3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:35.611387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:35.736717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5bd2d-8c98-4fc7-b134-26b430489123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:35.737344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5bd2d-8c98-4fc7-b134-26b430489123 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.737691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5bd2d-8c98-4fc7-b134-26b430489123 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.737869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5bd2d-8c98-4fc7-b134-26b430489123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.840702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:35.874562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 854/1705] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:35.892650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:35.895187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:35.925434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0fe4a-3d6b-4374-b91c-6451fb7bd2aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:35.954612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01cbaf-f95e-401a-86b0-29575c46b5dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:35.955414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01cbaf-f95e-401a-86b0-29575c46b5dc service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:35.955786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01cbaf-f95e-401a-86b0-29575c46b5dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:35.962646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01cbaf-f95e-401a-86b0-29575c46b5dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:35.981977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5bd2d-8c98-4fc7-b134-26b430489123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:35.984203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5bd2d-8c98-4fc7-b134-26b430489123 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:35.999656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01cbaf-f95e-401a-86b0-29575c46b5dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:36.002341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01cbaf-f95e-401a-86b0-29575c46b5dc service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:36.013800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 852/1706] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:36.025716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 855/1707] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:36.081905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:36.221863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94190906-c01d-4abe-b85e-e8f400a93df9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:36.222309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94190906-c01d-4abe-b85e-e8f400a93df9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:36.222895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94190906-c01d-4abe-b85e-e8f400a93df9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:36.230793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 853/1708] 10.222.0.78 () {54 vars in 961 bytes} [Wed Aug 10 23:13:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:36.236340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:36.236635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:36.236936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:36.282954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:36.286369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:36.357814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3d5a9-5f3a-47f5-9d6c-0a27cb0a93df None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:36.357814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3d5a9-5f3a-47f5-9d6c-0a27cb0a93df None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:36.357814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3d5a9-5f3a-47f5-9d6c-0a27cb0a93df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:36.398610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c2a3d5a9-5f3a-47f5-9d6c-0a27cb0a93df None None] MFA Rules not processed for user `ffb46379ef204df6bebd58f3bdb78ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:36.407179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c2a3d5a9-5f3a-47f5-9d6c-0a27cb0a93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:36.471551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 856/1709] 10.222.0.78 () {58 vars in 1058 bytes} [Wed Aug 10 23:13:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:36.472987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:36.497534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcd0a1-d4f4-4148-893e-f6035d41f9ee None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:36.521922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:36.522557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:36.527950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:36.528420 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:36.533222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:36.560334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 854/1710] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:36.564031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:36.573947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a215a-b09b-4600-bd78-484a02fc4809 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:36.615213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 857/1711] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:36.700537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:36.700834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:36.701122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:36.769485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:36.801524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] MFA Rules not processed for user `c5f2e6ec64db4ba3b3db2ca564c26bed`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:36.801524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:36.995638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 855/1712] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:37.008969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:37.009434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:37.009561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:37.036056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] MFA Rules not processed for user `c5f2e6ec64db4ba3b3db2ca564c26bed`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:37.037729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:37.050291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 858/1713] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:37.064039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:37.180031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03a057-e298-4551-af0f-70420f0b6fcc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:37.206003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f0515-9f35-45fb-b237-7e2e44981377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:37.211342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f0515-9f35-45fb-b237-7e2e44981377 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:37.211342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f0515-9f35-45fb-b237-7e2e44981377 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:37.211342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f0515-9f35-45fb-b237-7e2e44981377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:37.231308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f0515-9f35-45fb-b237-7e2e44981377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:37.235374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f0515-9f35-45fb-b237-7e2e44981377 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:37.276170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 856/1714] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:37.540737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:37.855593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:38.120141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:38.284770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7926ff-bce6-4577-a7c4-fc936b3657e9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:38.290999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:38.298724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:38.298724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:38.298724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] PATH_INFO: `/v3/users/6d18d72df0754cddb04c40eca228227e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:38.298724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d18d72df0754cddb04c40eca228227e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:38.299120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c03b1-392c-416f-a3a7-57b20d4cb87b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:38.299120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c03b1-392c-416f-a3a7-57b20d4cb87b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:38.299120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c03b1-392c-416f-a3a7-57b20d4cb87b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:38.299732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:38.299832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 857/1715] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 10 23:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:38.304902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:38.305269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:38.305757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:38.401309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:38.426946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:38.431362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:38.644818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d18d72df0754cddb04c40eca228227e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:38.725736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 858/1716] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:13:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:38.725736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:38.731506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bdd13-e017-4f5b-bd85-1764b538ca4e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:38.736339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e49d6a-7856-42c2-ae00-b3747282d10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:38.740574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d18d72df0754cddb04c40eca228227e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:38.747778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e49d6a-7856-42c2-ae00-b3747282d10f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:38.748348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e49d6a-7856-42c2-ae00-b3747282d10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:38.748812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e49d6a-7856-42c2-ae00-b3747282d10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:38.779961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d18d72df0754cddb04c40eca228227e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:38.784451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d18d72df0754cddb04c40eca228227e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:38.785506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e49d6a-7856-42c2-ae00-b3747282d10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:38.786881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d18d72df0754cddb04c40eca228227e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:38.787842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e49d6a-7856-42c2-ae00-b3747282d10f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:38.808313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 859/1717] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:13:38.888622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 859/1718] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:38 2022] DELETE /identity/v3/users/6d18d72df0754cddb04c40eca228227e => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:38.890204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122b9b-d4fc-46b8-ab65-6df3a220dd4b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:38.890578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122b9b-d4fc-46b8-ab65-6df3a220dd4b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:38.890945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122b9b-d4fc-46b8-ab65-6df3a220dd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:38.936358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-87122b9b-d4fc-46b8-ab65-6df3a220dd4b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:38.937594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-87122b9b-d4fc-46b8-ab65-6df3a220dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:39.187884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 860/1719] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:39.209848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:39.211832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:39.373797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d712-9933-4173-8043-19da908e69fb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:39.374754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d712-9933-4173-8043-19da908e69fb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:39.375038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d712-9933-4173-8043-19da908e69fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:39.383750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab865342-adc6-46cc-ac10-9c362e6fee2e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:39.395439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309e914-ec05-4b43-a4c5-fdd1f8199725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:39.403015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309e914-ec05-4b43-a4c5-fdd1f8199725 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:39.403015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309e914-ec05-4b43-a4c5-fdd1f8199725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:39.403015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309e914-ec05-4b43-a4c5-fdd1f8199725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:39.414258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309e914-ec05-4b43-a4c5-fdd1f8199725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:39.415890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309e914-ec05-4b43-a4c5-fdd1f8199725 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:39.418921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7d05d712-9933-4173-8043-19da908e69fb None None] MFA Rules not processed for user `45dd5b35a039468d99ec0d393056c5af`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:39.422053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7d05d712-9933-4173-8043-19da908e69fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:39.449574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 860/1720] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:39.551534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 861/1721] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:40.005166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:40.731615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:40.732457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:40.733055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:40.757299 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] MFA Rules not processed for user `22e92975af1d439f88e33cc494c88b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:40.759109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:40.818830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d712-9933-4173-8043-19da908e69fb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:40.822562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7d05d712-9933-4173-8043-19da908e69fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:40.836510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:40.860259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d712-9933-4173-8043-19da908e69fb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:40.876386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:40.878754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 861/1722] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:40.878859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:40.878859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:40.878859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] PATH_INFO: `/v3/projects/9dedd403a353497cba9656dcb56b1cfb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:40.880657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dedd403a353497cba9656dcb56b1cfb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:40.882773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:41.095049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dedd403a353497cba9656dcb56b1cfb', 'request_id': 'req-7cd22d20-4abf-4d50-86f4-6f00d340e230'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.203931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dedd403a353497cba9656dcb56b1cfb', 'request_id': 'req-7cd22d20-4abf-4d50-86f4-6f00d340e230'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.210860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dedd403a353497cba9656dcb56b1cfb', 'request_id': 'req-7cd22d20-4abf-4d50-86f4-6f00d340e230'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.240239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] The token cache is being invalidate because project 9dedd403a353497cba9656dcb56b1cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:41.240788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.241593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.243636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 862/1723] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:40 2022] DELETE /identity/v3/projects/9dedd403a353497cba9656dcb56b1cfb => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:13:41.248712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:41.265476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a3d0b-b561-4d73-87d6-5735bb290e37 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:41.355324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:41.356346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:41.356816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:41.357416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] PATH_INFO: `/v3/users/19c7b58c350946b592acd469b17ca3df` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:41.358901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c7b58c350946b592acd469b17ca3df)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:41.361288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:41.629052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c7b58c350946b592acd469b17ca3df'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.642952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c7b58c350946b592acd469b17ca3df'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.659683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c7b58c350946b592acd469b17ca3df'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.660886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c7b58c350946b592acd469b17ca3df'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.661797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1799498e-32fa-41b0-9d1c-f09ebf73cbf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c7b58c350946b592acd469b17ca3df'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:41.808370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 862/1724] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:41 2022] DELETE /identity/v3/users/19c7b58c350946b592acd469b17ca3df => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:42.389061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:42.391183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:42.407462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22d20-4abf-4d50-86f4-6f00d340e230 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:42.439643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:42.478858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:42.479655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:42.480122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:42.480398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] PATH_INFO: `/v3/projects/f59447b4b9b7468f9c6b11ddfd3e8bb1` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:42.483506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59447b4b9b7468f9c6b11ddfd3e8bb1)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:42.485977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:42.638991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59447b4b9b7468f9c6b11ddfd3e8bb1', 'request_id': 'req-edab8b1b-ec3f-4851-a999-04d1f2145a9b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:42.770789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59447b4b9b7468f9c6b11ddfd3e8bb1', 'request_id': 'req-edab8b1b-ec3f-4851-a999-04d1f2145a9b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:42.781478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59447b4b9b7468f9c6b11ddfd3e8bb1', 'request_id': 'req-edab8b1b-ec3f-4851-a999-04d1f2145a9b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:42.799747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] The token cache is being invalidate because project f59447b4b9b7468f9c6b11ddfd3e8bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:42.800059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:42.802091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-edab8b1b-ec3f-4851-a999-04d1f2145a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:42.805041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 863/1725] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:42 2022] DELETE /identity/v3/projects/f59447b4b9b7468f9c6b11ddfd3e8bb1 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:42.811265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:42.811265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:42.812283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:42.835254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:42.839999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:42.919379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 863/1726] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:42.928831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:42.930368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:42.930903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:42.964167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:42.965393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:42.975300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 864/1727] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:42.979356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:42.990267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80baeedf-fe29-4aa3-b58b-5fda0879f81c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:42.995924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:42.996635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:42.996915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:42.997181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:42.997824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:42.998122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:42.999759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.012703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 864/1728] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:13:43.017605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:43.027390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4d45-9aad-4ddb-8db5-d2f87432a49b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:43.032335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:43.032942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.033219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.033487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.034115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:43.036192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.117458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 865/1729] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:43 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:43.118526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:43.129033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a870b-c434-4232-bb98-31d3408a73dd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:43.207675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:43.208383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.208742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.209905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.210556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:43.213713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.328150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 865/1730] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:43 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:13:43.332394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:43.345775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e008e-827d-4101-84f5-7bcba0d3c32a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:43.356307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:43.357244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.357517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.357961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.358604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:43.360312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.384866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 866/1731] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:43.389163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:43.404023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4295c-4568-461b-80f9-0d7756b255db admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:43.412541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec4481-b8bd-42b8-a5dc-238678d0c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:43.414485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec4481-b8bd-42b8-a5dc-238678d0c856 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.414821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec4481-b8bd-42b8-a5dc-238678d0c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.415100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec4481-b8bd-42b8-a5dc-238678d0c856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.415812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec4481-b8bd-42b8-a5dc-238678d0c856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:43.418603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec4481-b8bd-42b8-a5dc-238678d0c856 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.437019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 866/1732] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:13:43.440978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:43.467023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daf40-5bc6-47a6-8c61-542df28e84b0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:43.474174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:43.475255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.475877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.476307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] PATH_INFO: `/v3/projects/27922d1d043047d0bb3cac722895ae62/users/35aa4ae031a140e2b68fff154343de1a/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.501888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27922d1d043047d0bb3cac722895ae62, user_id=35aa4ae031a140e2b68fff154343de1a, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:43.505961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.604014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 867/1733] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:43 2022] PUT /identity/v3/projects/27922d1d043047d0bb3cac722895ae62/users/35aa4ae031a140e2b68fff154343de1a/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:13:43.615260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.616632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.617389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.661872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] MFA Rules not processed for user `35aa4ae031a140e2b68fff154343de1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:43.665033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:43.750250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 867/1734] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:43.754921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:43.776523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:43.826801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3564b22-1a18-4264-940c-80924f97cfe3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:43.831883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:43.832494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:43.832769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:43.833034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:43.833666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:43.837276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:43.992986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 868/1735] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:43 2022] POST /identity/v3/projects => generated 367 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:43.997105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.012523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b896-719d-4eb5-8ec8-845e44db209c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.089782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.090516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.090879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.091443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.093516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:44.102649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:44.215605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 868/1736] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:43 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:44.219539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.230867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35287b84-e6a6-4a8c-b388-279a5c252f3f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.235839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.236281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.236561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.236856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.237530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:44.239193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:44.251159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 869/1737] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:44.255265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.269438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2bb69-b17f-42bb-89c1-044db5c854a6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.274166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.274937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.275319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.275666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] PATH_INFO: `/v3/projects/8ab8660a6a91408590cfd58a04107254/users/ba03f88a6b3d4dd0a614e089de1b81e2/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.293775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab8660a6a91408590cfd58a04107254, user_id=ba03f88a6b3d4dd0a614e089de1b81e2, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:44.296500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:44.434975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 869/1738] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:44 2022] PUT /identity/v3/projects/8ab8660a6a91408590cfd58a04107254/users/ba03f88a6b3d4dd0a614e089de1b81e2/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:44.440419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.455427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfac76-b6c2-4861-a0e3-96e7c949d72f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.523368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8db7c-d9f7-4dd6-a6e0-05c2cc20f925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.524030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8db7c-d9f7-4dd6-a6e0-05c2cc20f925 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.524308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8db7c-d9f7-4dd6-a6e0-05c2cc20f925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.524602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8db7c-d9f7-4dd6-a6e0-05c2cc20f925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.525351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8db7c-d9f7-4dd6-a6e0-05c2cc20f925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:44.527054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8db7c-d9f7-4dd6-a6e0-05c2cc20f925 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:44.542766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 870/1739] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:44.545767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.558497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce15558-5d44-4890-81aa-fa5dd23ba6cb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.563149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.563804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.564080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.564530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] PATH_INFO: `/v3/projects/8ab8660a6a91408590cfd58a04107254/users/ba03f88a6b3d4dd0a614e089de1b81e2/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.566564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab8660a6a91408590cfd58a04107254, user_id=ba03f88a6b3d4dd0a614e089de1b81e2, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:44.569270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:44.579452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 870/1740] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:44 2022] PUT /identity/v3/projects/8ab8660a6a91408590cfd58a04107254/users/ba03f88a6b3d4dd0a614e089de1b81e2/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:13:44.589639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.590057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.590231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.624239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] MFA Rules not processed for user `ba03f88a6b3d4dd0a614e089de1b81e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:44.625523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:44.721543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 871/1741] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:44.734886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.737017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:44.795240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6562251-8cfa-4b09-adae-22ae63122f8f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.798891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecdc2e-859f-4d61-b720-475773914558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.799494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecdc2e-859f-4d61-b720-475773914558 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.799860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecdc2e-859f-4d61-b720-475773914558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.800142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecdc2e-859f-4d61-b720-475773914558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:44.809751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecdc2e-859f-4d61-b720-475773914558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:44.813093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecdc2e-859f-4d61-b720-475773914558 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:44.821695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 871/1742] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:44.950690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:44.965378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5736ec-5d0a-4d04-a265-90d7877bcf69 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:44.969360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050840-8591-4b22-90d5-579d45e035eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:44.970025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050840-8591-4b22-90d5-579d45e035eb service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:44.970306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050840-8591-4b22-90d5-579d45e035eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:44.970865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050840-8591-4b22-90d5-579d45e035eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:45.042452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:45.112762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050840-8591-4b22-90d5-579d45e035eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:45.114094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050840-8591-4b22-90d5-579d45e035eb service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:45.150058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 872/1743] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:46.595841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:47.768849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:49.602082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:49.606328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:49.606328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:49.650551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:49.651644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:49.651644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:49.659976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:49.663721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:49.793366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 872/1744] 10.222.0.89 () {58 vars in 1015 bytes} [Wed Aug 10 23:13:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:49.810938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:49.814433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:49.828937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:49.830573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:49.859594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011f7e-9164-4b8d-80ba-b3463ee9de0b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:49.904637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:49.922239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:49.922239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:49.922239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:49.951558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:49.955350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:49.993812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 873/1745] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:13:49.996749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:50.000548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:50.025605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 873/1746] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:13:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:50.059035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f5777-8116-469c-a622-fd6d515c291a service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:50.076578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:50.081258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:50.081693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:50.082475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:50.092522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:50.094631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:50.106253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 874/1747] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:50.391120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:50.392745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:50.617969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cedcc-dbdc-4e03-aff2-44e2009b54b0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:50.629864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf34fa5-fc9d-4e02-9306-c8cec8b214a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:50.640885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34fa5-fc9d-4e02-9306-c8cec8b214a4 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:50.640885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34fa5-fc9d-4e02-9306-c8cec8b214a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:50.640885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34fa5-fc9d-4e02-9306-c8cec8b214a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:50.643880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf34fa5-fc9d-4e02-9306-c8cec8b214a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:50.645905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf34fa5-fc9d-4e02-9306-c8cec8b214a4 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:50.658097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 874/1748] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:13:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:13:51.612215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:51.701314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:51.756476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd327e2-64eb-4b79-ad49-181eb46df33b service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:51.774899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:51.775745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:51.776065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:51.776562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] PATH_INFO: `/v3/users/35aa4ae031a140e2b68fff154343de1a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:51.783544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35aa4ae031a140e2b68fff154343de1a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:51.785864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:52.182764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35aa4ae031a140e2b68fff154343de1a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:52.233307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35aa4ae031a140e2b68fff154343de1a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:52.257769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35aa4ae031a140e2b68fff154343de1a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:52.259314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35aa4ae031a140e2b68fff154343de1a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:52.261391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35aa4ae031a140e2b68fff154343de1a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:52.569074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 875/1749] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:51 2022] DELETE /identity/v3/users/35aa4ae031a140e2b68fff154343de1a => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:52.604514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47e157-af96-45e5-973c-059d3ce2e47e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:52.605642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47e157-af96-45e5-973c-059d3ce2e47e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:52.606148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47e157-af96-45e5-973c-059d3ce2e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:52.648274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9e47e157-af96-45e5-973c-059d3ce2e47e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:52.652263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9e47e157-af96-45e5-973c-059d3ce2e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:52.766769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:52.880093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 875/1750] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:52.956929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:52.963470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:53.060795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadfb8c-8db0-45d3-8284-b24ea7d7c172 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:53.070022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:53.078892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:53.078892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:53.078892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:53.083586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234f62-b435-4b11-8cd6-c8cb55b6a2d0 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:53.083875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234f62-b435-4b11-8cd6-c8cb55b6a2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:53.084174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234f62-b435-4b11-8cd6-c8cb55b6a2d0 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:53.085563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 876/1751] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:13:53.107245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:53.111883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:53.135556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 876/1752] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:53.155437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:53.156016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:53.156473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:53.208979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:53.217116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:53.368316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 877/1753] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:13:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:13:53.425687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:53.427829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:53.450442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614fe0d-92b4-400e-931d-adeaf33d0a7d service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:53.460605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056788-cb3c-4e7c-bcb8-654ea8a04342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:53.462280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056788-cb3c-4e7c-bcb8-654ea8a04342 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:53.462988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056788-cb3c-4e7c-bcb8-654ea8a04342 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:53.464014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056788-cb3c-4e7c-bcb8-654ea8a04342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:53.482672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a056788-cb3c-4e7c-bcb8-654ea8a04342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:53.484663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a056788-cb3c-4e7c-bcb8-654ea8a04342 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:53.504087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 877/1754] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:54.691642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:54.693659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:54.727914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2bbbf-16aa-4324-84d5-1c932b93fa08 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:54.733553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:54.734187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:54.734484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:54.734786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] PATH_INFO: `/v3/projects/27922d1d043047d0bb3cac722895ae62` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:54.736016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27922d1d043047d0bb3cac722895ae62)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:54.738115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:54.884551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27922d1d043047d0bb3cac722895ae62', 'request_id': 'req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:54.902685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363e00a-d37b-45c7-a85d-84bd8b9dfbea None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:54.904051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363e00a-d37b-45c7-a85d-84bd8b9dfbea None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:54.904051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363e00a-d37b-45c7-a85d-84bd8b9dfbea None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:54.904887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 878/1755] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 23:13:54.930187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:54.933436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:54.933436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:54.951507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:54.952832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:54.964620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27922d1d043047d0bb3cac722895ae62', 'request_id': 'req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:54.975435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27922d1d043047d0bb3cac722895ae62', 'request_id': 'req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.007512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] The token cache is being invalidate because project 27922d1d043047d0bb3cac722895ae62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:55.008117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.009305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.026712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 878/1756] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:54 2022] DELETE /identity/v3/projects/27922d1d043047d0bb3cac722895ae62 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:55.031267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:55.031369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:55.031369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:55.045690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 879/1757] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:13:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:55.058436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:55.063145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:55.074033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac89155-2a89-4e66-9b28-b39982480b6a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:55.076448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:55.080014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:55.080623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:55.080901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:55.081201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:55.092061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:55.093293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d20be-b1ac-4c86-9fe4-095ed1e4d155 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:55.093672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:55.098183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:55.098836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:55.099112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:55.099382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] PATH_INFO: `/v3/users/ba03f88a6b3d4dd0a614e089de1b81e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:55.100558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba03f88a6b3d4dd0a614e089de1b81e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:55.102444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:55.106039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 880/1758] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:55.386871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba03f88a6b3d4dd0a614e089de1b81e2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.397403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba03f88a6b3d4dd0a614e089de1b81e2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.405587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba03f88a6b3d4dd0a614e089de1b81e2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.406340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba03f88a6b3d4dd0a614e089de1b81e2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.406975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5eb4b99a-e2b0-4366-9c08-92193fdf65b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba03f88a6b3d4dd0a614e089de1b81e2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:55.527306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 879/1759] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:55 2022] DELETE /identity/v3/users/ba03f88a6b3d4dd0a614e089de1b81e2 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:55.952338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:55.966400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dd9a4-5b58-4d2a-a7d6-abcbbb8945a5 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.054216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.055063 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.055344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.055625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] PATH_INFO: `/v3/projects/8ab8660a6a91408590cfd58a04107254` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.057451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab8660a6a91408590cfd58a04107254)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.060131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.186871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab8660a6a91408590cfd58a04107254', 'request_id': 'req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:56.260058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab8660a6a91408590cfd58a04107254', 'request_id': 'req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:56.268271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab8660a6a91408590cfd58a04107254', 'request_id': 'req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:56.282634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] The token cache is being invalidate because project 8ab8660a6a91408590cfd58a04107254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:56.285872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:56.285872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7066fec5-0d9a-41bb-aeb6-8c946bdabf0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:56.285872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 881/1760] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:55 2022] DELETE /identity/v3/projects/8ab8660a6a91408590cfd58a04107254 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:56.292161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.292817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.292817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.314177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:56.315967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:56.376360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 880/1761] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:56.384130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.384416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.384718 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.404469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:56.406948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:56.413455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 882/1762] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:56.418232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:56.430630 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c105630-4606-4597-8a0a-b683c2748430 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.437191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.437825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.438188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.438474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.439155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:13:56.439455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.441044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.456839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 881/1763] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:13:56.458732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:56.468258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c3ab4-8d3a-4da0-a8b4-5a57cef11db7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.473101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.473700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.473976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.474247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.474904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.476791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.587639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 883/1764] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:56 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:56.592238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:56.602144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a66755-08ae-496e-a509-0ed67da1e4a5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.657946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.658474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.658787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.659059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.659694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.661660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.734875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 882/1765] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:56 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:56.738987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:56.749325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2df3-ef93-4c39-a9fb-e83f9f0a7755 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.754118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.754877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.755164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.755494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.756125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.757703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.768106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 884/1766] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:56.772069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:56.782454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8705b4c-c736-490d-8e6b-1b4ba7bc6f3c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.787237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd2d0b-f690-4ff8-aafc-e5426b844548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.787831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2d0b-f690-4ff8-aafc-e5426b844548 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.788115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2d0b-f690-4ff8-aafc-e5426b844548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.788389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2d0b-f690-4ff8-aafc-e5426b844548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.789024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd2d0b-f690-4ff8-aafc-e5426b844548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.790676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd2d0b-f690-4ff8-aafc-e5426b844548 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.801288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 883/1767] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:56.805622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:56.815606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f000f6c-5417-4138-b565-674a82abed90 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:56.820336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:56.820990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.821265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.821536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] PATH_INFO: `/v3/projects/b193a70f429447db94b5a5ded4212158/users/6c8a6864441f489f82b3ce345b0a4ed8/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.836655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b193a70f429447db94b5a5ded4212158, user_id=6c8a6864441f489f82b3ce345b0a4ed8, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:56.840901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:56.915560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 885/1768] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:56 2022] PUT /identity/v3/projects/b193a70f429447db94b5a5ded4212158/users/6c8a6864441f489f82b3ce345b0a4ed8/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:56.925796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:56.926083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:56.926357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:56.954898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] MFA Rules not processed for user `6c8a6864441f489f82b3ce345b0a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:56.956880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:57.019857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 884/1769] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:57.024377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.098700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bafdf-94a6-4bd0-928b-8f94874b8442 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.103509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.104176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.104453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.104721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.105385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.107339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.216793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 886/1770] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:13:57 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:13:57.217951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.229297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f6753-74b0-430b-bc5e-b6e051117770 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.290224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.292790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.292790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.292790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.295869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.295869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.315806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:57.443138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 885/1771] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:13:57 2022] POST /identity/v3/users => generated 299 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:13:57.446801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.457340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f67f41-829c-445c-8837-48d29504dcee admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.463505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.464120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.464408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.464678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.465367 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.467428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.480201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 887/1772] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:57.486874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.500820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b061-b985-4645-8cb4-6812f2db4523 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.507929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.508203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.508741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.509071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] PATH_INFO: `/v3/projects/baca573a92ea49ed98b13a0ae88926ad/users/f0d58023ac924e6d817a284daceda843/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.525388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=baca573a92ea49ed98b13a0ae88926ad, user_id=f0d58023ac924e6d817a284daceda843, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.528243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.561516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:57.596544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 886/1773] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:57 2022] PUT /identity/v3/projects/baca573a92ea49ed98b13a0ae88926ad/users/f0d58023ac924e6d817a284daceda843/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:57.600870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.612447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29a9ee-240a-462e-88df-916dec9a559e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.671583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd8de6-2f87-4f2e-a27c-81599f7fedfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.672229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8de6-2f87-4f2e-a27c-81599f7fedfa admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.672510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8de6-2f87-4f2e-a27c-81599f7fedfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.672789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8de6-2f87-4f2e-a27c-81599f7fedfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.673423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd8de6-2f87-4f2e-a27c-81599f7fedfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.675089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd8de6-2f87-4f2e-a27c-81599f7fedfa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.684930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 888/1774] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:13:57.689282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.699665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabd6aa-3854-475d-be03-3605c100e30c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.704338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.704980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.705260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.705532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] PATH_INFO: `/v3/projects/baca573a92ea49ed98b13a0ae88926ad/users/f0d58023ac924e6d817a284daceda843/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.707451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baca573a92ea49ed98b13a0ae88926ad, user_id=f0d58023ac924e6d817a284daceda843, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.710134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.721257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 887/1775] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:13:57 2022] PUT /identity/v3/projects/baca573a92ea49ed98b13a0ae88926ad/users/f0d58023ac924e6d817a284daceda843/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:57.733227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.733508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.733779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.760961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] MFA Rules not processed for user `f0d58023ac924e6d817a284daceda843`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:57.762234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:57.828011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 889/1776] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:57.839880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:57.841679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:57.897784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30170b5c-aacd-41b9-9bdd-d722f95bf043 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:57.902191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bb509-0942-4fdd-b1e7-d5c6b4f6124a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:57.902831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bb509-0942-4fdd-b1e7-d5c6b4f6124a service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:57.903107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bb509-0942-4fdd-b1e7-d5c6b4f6124a service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:57.903377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bb509-0942-4fdd-b1e7-d5c6b4f6124a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:57.912719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bb509-0942-4fdd-b1e7-d5c6b4f6124a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:57.914155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bb509-0942-4fdd-b1e7-d5c6b4f6124a service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:57.924815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 888/1777] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:57.952695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:58.016909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2283cd6-c308-4aae-9405-86272d5d83c6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:58.023264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:58.024103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:58.024423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:58.025117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] PATH_INFO: `/v3/users/6c8a6864441f489f82b3ce345b0a4ed8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:58.026513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8a6864441f489f82b3ce345b0a4ed8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:58.028543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:58.251086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8a6864441f489f82b3ce345b0a4ed8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:58.263275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8a6864441f489f82b3ce345b0a4ed8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:58.270337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8a6864441f489f82b3ce345b0a4ed8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:58.272663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8a6864441f489f82b3ce345b0a4ed8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:58.273297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8a6864441f489f82b3ce345b0a4ed8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:58.381244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 890/1778] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:57 2022] DELETE /identity/v3/users/6c8a6864441f489f82b3ce345b0a4ed8 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:58.386444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba672-9e83-4287-984b-efb496825e42 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:58.386844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba672-9e83-4287-984b-efb496825e42 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:58.387070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba672-9e83-4287-984b-efb496825e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:58.407878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-69aba672-9e83-4287-984b-efb496825e42 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:13:58.409087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-69aba672-9e83-4287-984b-efb496825e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:58.477690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 889/1779] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:13:58.484263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:58.485757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:58.545102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5022a9d-3634-4a85-9a31-97cbe20b9781 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:58.550247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:58.551167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:58.551529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:58.552320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:58.566398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:58.567201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:58.579545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 891/1780] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:13:58.786778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:58.855241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:13:59.131574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aba672-9e83-4287-984b-efb496825e42 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:59.135346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-69aba672-9e83-4287-984b-efb496825e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:13:59.153871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aba672-9e83-4287-984b-efb496825e42 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:59.160819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:59.162567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:59.162962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:59.163239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] PATH_INFO: `/v3/projects/b193a70f429447db94b5a5ded4212158` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:59.164642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b193a70f429447db94b5a5ded4212158)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:59.167531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:13:59.528402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b193a70f429447db94b5a5ded4212158', 'request_id': 'req-1d562c33-581f-4d3b-ade8-575f679b1e8a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:59.775881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b193a70f429447db94b5a5ded4212158', 'request_id': 'req-1d562c33-581f-4d3b-ade8-575f679b1e8a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:59.781993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b193a70f429447db94b5a5ded4212158', 'request_id': 'req-1d562c33-581f-4d3b-ade8-575f679b1e8a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:59.797976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] The token cache is being invalidate because project b193a70f429447db94b5a5ded4212158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:13:59.798779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:59.799681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:13:59.801783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 890/1781] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:13:59 2022] DELETE /identity/v3/projects/b193a70f429447db94b5a5ded4212158 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:13:59.806064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:13:59.819365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32449c7-a34a-4f97-8b09-7bb1090550d6 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:13:59.896049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:13:59.896810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:13:59.897116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:13:59.897390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] PATH_INFO: `/v3/users/f0d58023ac924e6d817a284daceda843` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:13:59.898529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d58023ac924e6d817a284daceda843)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:13:59.901100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:00.288650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d58023ac924e6d817a284daceda843'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:00.307779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d58023ac924e6d817a284daceda843'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:00.309462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d58023ac924e6d817a284daceda843'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:00.310228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d58023ac924e6d817a284daceda843'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:00.310871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df29fdbe-4d02-4ff5-a3e4-f8686e8eee1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d58023ac924e6d817a284daceda843'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:00.497797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 892/1782] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:13:59 2022] DELETE /identity/v3/users/f0d58023ac924e6d817a284daceda843 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:00.580477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:00.582290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:00.598414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562c33-581f-4d3b-ade8-575f679b1e8a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:00.744135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:00.744135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:00.744135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:00.744135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] PATH_INFO: `/v3/users/a24422d1f1e54ca5beff6b929b69a062` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:00.745593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24422d1f1e54ca5beff6b929b69a062)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:00.745593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:01.092944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:01.170416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:01.253026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24422d1f1e54ca5beff6b929b69a062'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:01.274814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24422d1f1e54ca5beff6b929b69a062'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:01.284659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24422d1f1e54ca5beff6b929b69a062'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:01.285453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24422d1f1e54ca5beff6b929b69a062'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:01.286322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24422d1f1e54ca5beff6b929b69a062'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:01.409024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 891/1783] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:00 2022] DELETE /identity/v3/users/a24422d1f1e54ca5beff6b929b69a062 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:01.414868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:01.415392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:01.415819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:01.452547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:01.454276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:01.576482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 893/1784] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:01.581531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:01.583058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:01.637898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:01.639546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:01.656841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab6c71-a9c2-4c93-ac9e-142c6a7517fa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:01.672752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:01.672752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:01.672752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:01.673236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] PATH_INFO: `/v3/projects/baca573a92ea49ed98b13a0ae88926ad` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:01.674805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=baca573a92ea49ed98b13a0ae88926ad)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:01.677769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:01.690441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229818b6-e701-4e78-a72c-2b2399b0731b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:01.695952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce857a-fd05-4963-bc54-0d02dda61098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:01.696777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce857a-fd05-4963-bc54-0d02dda61098 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:01.697223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce857a-fd05-4963-bc54-0d02dda61098 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:01.699246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce857a-fd05-4963-bc54-0d02dda61098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:01.714257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce857a-fd05-4963-bc54-0d02dda61098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:01.715783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce857a-fd05-4963-bc54-0d02dda61098 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:01.726689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 892/1785] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:01.854269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baca573a92ea49ed98b13a0ae88926ad', 'request_id': 'req-946d8ad8-4252-4107-b1b3-e15d5633a4bc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:02.300481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baca573a92ea49ed98b13a0ae88926ad', 'request_id': 'req-946d8ad8-4252-4107-b1b3-e15d5633a4bc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:02.307985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baca573a92ea49ed98b13a0ae88926ad', 'request_id': 'req-946d8ad8-4252-4107-b1b3-e15d5633a4bc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:02.320373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] The token cache is being invalidate because project baca573a92ea49ed98b13a0ae88926ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:02.320659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:02.321323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-946d8ad8-4252-4107-b1b3-e15d5633a4bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:02.322940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 894/1786] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:01 2022] DELETE /identity/v3/projects/baca573a92ea49ed98b13a0ae88926ad => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:02.327858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:02.328148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:02.328446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:02.349082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:02.349850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:02.431146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 893/1787] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:02.443858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:02.444146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:02.444605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:02.481914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:02.499181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:02.515598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 895/1788] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:02.558515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:02.576069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43516c-bda5-44eb-b562-e44d667e8b72 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:02.593414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:02.606256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:02.606256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:02.606256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:02.606256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:02.606256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:02.606256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:02.630154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:02.630399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 894/1789] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:02.652342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957aaec-64a1-4339-94c8-9fff01c6db66 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:02.658649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:02.659304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:02.659889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:02.660164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:02.660816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:02.662907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:02.874322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:02.874322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:02.892144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 896/1790] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:02 2022] POST /identity/v3/projects => generated 401 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:02.899809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8d085-8641-400e-a442-2cf71ecee8f5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:02.903305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:03.032859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:03.038381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.038381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.038381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] PATH_INFO: `/v3/projects/b0b3e39661374044a2aac6dd8a912af7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:03.038381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b3e39661374044a2aac6dd8a912af7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:03.038381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:03.042755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ae167-e730-47ed-9e76-e636f10c211e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:03.052227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:03.054114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.054854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.055514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:03.059188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:03.065008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:03.452903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 897/1791] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:02 2022] POST /identity/v3/users => generated 301 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:03.457122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:03.464724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b3e39661374044a2aac6dd8a912af7', 'request_id': 'req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:03.473315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3dfe-4c32-4a01-9e75-e3bbddd5b2dc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:03.544724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:03.545912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.546536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.546995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:03.547902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:03.550125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:03.564958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 898/1792] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:03.569525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:03.582630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74ddbe-c431-4b55-85f8-1d4e0ec69dd7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:03.595778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:03.595778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.596249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.597668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:03.597668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:03.599345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:03.616160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 899/1793] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:03.621268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:03.633940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec85908-bbed-4341-8815-a661f9c3833e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:03.639729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0916f1-4dfb-47e3-9e05-e87708a1a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:03.640419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0916f1-4dfb-47e3-9e05-e87708a1a11e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.640699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0916f1-4dfb-47e3-9e05-e87708a1a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.640971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0916f1-4dfb-47e3-9e05-e87708a1a11e admin admin] PATH_INFO: `/v3/projects/b52f61cda36f4d8382b4cb8d581dc19a/users/1373fd96ef1e49789ecf827b24c9af4b/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:03.660655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0916f1-4dfb-47e3-9e05-e87708a1a11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52f61cda36f4d8382b4cb8d581dc19a, user_id=1373fd96ef1e49789ecf827b24c9af4b, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:03.663611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0916f1-4dfb-47e3-9e05-e87708a1a11e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:03.694917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b3e39661374044a2aac6dd8a912af7', 'request_id': 'req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:03.704158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b3e39661374044a2aac6dd8a912af7', 'request_id': 'req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:03.721876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] The token cache is being invalidate because project b0b3e39661374044a2aac6dd8a912af7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:03.722202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:03.722909 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:03.724872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 895/1794] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:02 2022] DELETE /identity/v3/projects/b0b3e39661374044a2aac6dd8a912af7 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:03.729276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:03.731286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:03.747375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ffc68d-ecf0-41ca-b335-b1a70c7e49ed admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:03.753515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:03.754903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.755416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.755953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] PATH_INFO: `/v3/users/695bb45a8051489a910f75acb505ac26` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:03.757907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695bb45a8051489a910f75acb505ac26)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:03.761659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:03.897037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 900/1795] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:03 2022] PUT /identity/v3/projects/b52f61cda36f4d8382b4cb8d581dc19a/users/1373fd96ef1e49789ecf827b24c9af4b/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:03.907619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:03.907885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:03.908157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:04.047997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] MFA Rules not processed for user `1373fd96ef1e49789ecf827b24c9af4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:04.050644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:04.061966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695bb45a8051489a910f75acb505ac26'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:04.086310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695bb45a8051489a910f75acb505ac26'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:04.101883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695bb45a8051489a910f75acb505ac26'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:04.106334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695bb45a8051489a910f75acb505ac26'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:04.106334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695bb45a8051489a910f75acb505ac26'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:04.210200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 901/1796] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:04.215541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:04.217037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:04.285294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f3dbc-316d-4885-962b-6a3bd7d47f48 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:04.290099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:04.290759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:04.291074 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:04.291347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:04.292068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:04.293908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:04.371982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 896/1797] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:03 2022] DELETE /identity/v3/users/695bb45a8051489a910f75acb505ac26 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:04.514077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 902/1798] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:04 2022] POST /identity/v3/projects => generated 399 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:04.518243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:04.579898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14df40-b99b-4205-b19e-b5d9d3ca9667 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:04.686065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:04.686699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:04.686983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:04.687253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:04.687889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:04.689915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:04.958736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 897/1799] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:04 2022] POST /identity/v3/users => generated 306 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:04.960934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:04.971875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a13db-1baa-4772-a2fd-c860df1b9716 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:04.976531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:04.977134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:04.977411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:04.977682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:04.978320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:04.980680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:04.996122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:04.998422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:05.017688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdca60a-2df5-46f5-bc72-354fa29c5df2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.026294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 903/1800] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:05.028391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.038037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.039031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.039858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.040338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] PATH_INFO: `/v3/projects/b681e409e9d4473ca2db0f72748c9930` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.043867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b681e409e9d4473ca2db0f72748c9930)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.048467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.060025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452f7bf-fa51-4452-90c1-0eab122cb019 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.070527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.071712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.072224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.072682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] PATH_INFO: `/v3/projects/9a789d18b10c4bb4b84b7d9607a67d43/users/76a07e6b5144468499af1c2be43921f6/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.089648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a789d18b10c4bb4b84b7d9607a67d43, user_id=76a07e6b5144468499af1c2be43921f6, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.105194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.345314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:05.354349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 904/1801] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:05 2022] PUT /identity/v3/projects/9a789d18b10c4bb4b84b7d9607a67d43/users/76a07e6b5144468499af1c2be43921f6/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:05.360366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.361215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b681e409e9d4473ca2db0f72748c9930', 'request_id': 'req-a9e75811-a499-424f-b5d5-102232488cc1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:05.447766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac129c9e-2b3c-4158-8dbc-57213763f257 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.465568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:05.470145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b681e409e9d4473ca2db0f72748c9930', 'request_id': 'req-a9e75811-a499-424f-b5d5-102232488cc1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:05.479356 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b681e409e9d4473ca2db0f72748c9930', 'request_id': 'req-a9e75811-a499-424f-b5d5-102232488cc1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:05.502985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] The token cache is being invalidate because project b681e409e9d4473ca2db0f72748c9930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:05.503285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:05.514004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a9e75811-a499-424f-b5d5-102232488cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:05.515984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 898/1802] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:04 2022] DELETE /identity/v3/projects/b681e409e9d4473ca2db0f72748c9930 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:05.530797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4db94a-c6ac-4409-b18c-111786bbf446 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.530797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4db94a-c6ac-4409-b18c-111786bbf446 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.530797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4db94a-c6ac-4409-b18c-111786bbf446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.546574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.547223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.547488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.547759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.548401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.550055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.553374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0d4db94a-c6ac-4409-b18c-111786bbf446 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:05.555765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0d4db94a-c6ac-4409-b18c-111786bbf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:05.571236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 905/1803] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:05.579192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 899/1804] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:05.581313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.581604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.581878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.583228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.585064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:05.598259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76958f94-3424-4559-bc00-dea407da1a4d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.602731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:05.603347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0832-9a04-417d-9956-9404dd9d867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.603965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:05.604228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0832-9a04-417d-9956-9404dd9d867f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.604578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0832-9a04-417d-9956-9404dd9d867f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.604938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0832-9a04-417d-9956-9404dd9d867f admin admin] PATH_INFO: `/v3/projects/9a789d18b10c4bb4b84b7d9607a67d43/users/76a07e6b5144468499af1c2be43921f6/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.607041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0832-9a04-417d-9956-9404dd9d867f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a789d18b10c4bb4b84b7d9607a67d43, user_id=76a07e6b5144468499af1c2be43921f6, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.609858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0832-9a04-417d-9956-9404dd9d867f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.611741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 900/1805] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:05.615989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.621370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 906/1806] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:05 2022] PUT /identity/v3/projects/9a789d18b10c4bb4b84b7d9607a67d43/users/76a07e6b5144468499af1c2be43921f6/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:05.626059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c90a7-c591-4388-8c0a-e06f79df6d34 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.636327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.636674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.636945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.664469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] MFA Rules not processed for user `76a07e6b5144468499af1c2be43921f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:05.665705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:05.681375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.682023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.682302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.682576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.683281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:05.683580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.685165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.698177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 901/1807] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:14:05.706439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.714562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f6272-57a4-4cf5-8e20-96f52b8934e8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.720048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.721045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.721381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.721870 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.722684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.724728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.745550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 907/1808] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:05.756162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.757534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:05.812978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716fe31-f824-4db3-8496-3d70958928c8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.816496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.817143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.817381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.817652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.827577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.831224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 902/1809] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:05 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:05.837426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:05.837547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:05.839943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 908/1810] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:05.895089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e40376-90d6-4795-843a-4ecc086cee0c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:05.899804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:05.900454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:05.900735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:05.901007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:05.901619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:05.903577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.102741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 903/1811] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:05 2022] POST /identity/v3/users => generated 284 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:06.110080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.121302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a65a94-71fb-468b-a30d-e903c4dadcfa service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.126505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.127442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.127724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.127996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.128687 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.130363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.149430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 909/1812] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:06.157454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.157668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.167956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee75ec0-a885-4992-ac33-9f81669658a7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.173271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.173869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.174142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.174410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.175112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.176706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.189731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 904/1813] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:06.194214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.204200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115480ee-58a6-44e9-9a6c-2b5f19abc9ad admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.208801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.209440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.209722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.209992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] PATH_INFO: `/v3/projects/a7556e22c0c34d548fb49d9dfac50dee/users/0441016b1eca46f0b333333454b9e2d4/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.214539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe91a6-b201-4267-9aa5-263df980ae35 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.218337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.219020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.219300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.219572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.221596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:06.226439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7556e22c0c34d548fb49d9dfac50dee, user_id=0441016b1eca46f0b333333454b9e2d4, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.229255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.280202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.281643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a5c2-4f35-4a82-8a9d-916ab521b503 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.290697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 910/1814] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:06 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:06.346312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 905/1815] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:06 2022] PUT /identity/v3/projects/a7556e22c0c34d548fb49d9dfac50dee/users/0441016b1eca46f0b333333454b9e2d4/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:06.357998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.358284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.358556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.386558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] MFA Rules not processed for user `0441016b1eca46f0b333333454b9e2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:06.387877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:06.457569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 911/1816] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:06.462067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.526935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809e5b4-265c-4551-b2c0-1a4d911c2279 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.531608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.532211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.532492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.532778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.533381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.535357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.674106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 906/1817] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:06 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:06.678240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.688741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40ee8d-0c9a-4024-b3d8-69567b6c7e97 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.758460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.759321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.759861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.760429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.761242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.763582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.889213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 912/1818] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:06 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:06.893167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.904266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391cc80-b810-4bee-a3de-f090be966b53 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.908839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.909425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.909707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.909984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.910767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.915390 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:06.923876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 907/1819] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:06.928081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:06.937904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060641f-944e-4c43-bec5-a0326bbedbda admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:06.942455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:06.943137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:06.944056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:06.944056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] PATH_INFO: `/v3/projects/ef60bbd1179a4f29823ee8d8b5db9bb1/users/02cbd2bd7f914df2a07e2ab3f55dfb7a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:06.957899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef60bbd1179a4f29823ee8d8b5db9bb1, user_id=02cbd2bd7f914df2a07e2ab3f55dfb7a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:06.960625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:07.032470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 913/1820] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:06 2022] PUT /identity/v3/projects/ef60bbd1179a4f29823ee8d8b5db9bb1/users/02cbd2bd7f914df2a07e2ab3f55dfb7a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:07.040776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:07.051933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e34c-6ecf-4c34-a864-883d052f3f40 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:07.111866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7439965-0d65-44e8-be5d-aed883934238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:07.112469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7439965-0d65-44e8-be5d-aed883934238 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:07.112746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7439965-0d65-44e8-be5d-aed883934238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:07.113013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7439965-0d65-44e8-be5d-aed883934238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:07.113740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7439965-0d65-44e8-be5d-aed883934238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:07.115433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7439965-0d65-44e8-be5d-aed883934238 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:07.126408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 908/1821] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:07.130742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:07.146742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204642-c947-424b-bf4c-7ed9764659a0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:07.147650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:07.149390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:07.149390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:07.149390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] PATH_INFO: `/v3/projects/ef60bbd1179a4f29823ee8d8b5db9bb1/users/02cbd2bd7f914df2a07e2ab3f55dfb7a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:07.152136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef60bbd1179a4f29823ee8d8b5db9bb1, user_id=02cbd2bd7f914df2a07e2ab3f55dfb7a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:07.154836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:07.174876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 914/1822] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:07 2022] PUT /identity/v3/projects/ef60bbd1179a4f29823ee8d8b5db9bb1/users/02cbd2bd7f914df2a07e2ab3f55dfb7a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:07.180591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:07.180783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:07.181067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:07.215067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] MFA Rules not processed for user `02cbd2bd7f914df2a07e2ab3f55dfb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:07.215067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:07.293251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 909/1823] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:07.301858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:07.304374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:07.366231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd05c4-71d5-49f0-95e4-8afd6d69c9ad admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:07.371217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069828fe-7426-4d13-9717-8cfde6397f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:07.371849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069828fe-7426-4d13-9717-8cfde6397f5a service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:07.372180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069828fe-7426-4d13-9717-8cfde6397f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:07.372451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069828fe-7426-4d13-9717-8cfde6397f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:07.383706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069828fe-7426-4d13-9717-8cfde6397f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:07.384922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069828fe-7426-4d13-9717-8cfde6397f5a service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:07.398916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 915/1824] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:07.528793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:07.630202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:08.391857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:08.545848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=5,keystone:DELETE=8,keystone:SELECT=87 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:08.824738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:08.997169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:09.043771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:10.575599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:10.596699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:10.640877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ef77-4c41-4515-a8e3-4f016099ba3e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:10.645498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:10.646193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:10.646540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:10.647640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] PATH_INFO: `/v3/users/0441016b1eca46f0b333333454b9e2d4` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:10.649321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0441016b1eca46f0b333333454b9e2d4)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:10.652058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:10.833982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0441016b1eca46f0b333333454b9e2d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:10.846994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0441016b1eca46f0b333333454b9e2d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:10.854034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0441016b1eca46f0b333333454b9e2d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:10.854800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0441016b1eca46f0b333333454b9e2d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:10.855438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0441016b1eca46f0b333333454b9e2d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:10.927859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 910/1825] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:10 2022] DELETE /identity/v3/users/0441016b1eca46f0b333333454b9e2d4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:10.932816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:10.933338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:10.933338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:10.957295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:10.958474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:11.019380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:11.028903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 916/1826] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:11.039657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:11.041710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:11.111665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62a0f6-feeb-467b-9846-9deb91a302a2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:11.115584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:11.116415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:11.116880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:11.117407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:11.138125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:11.143894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:11.158208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 911/1827] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:12.599390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:12.600930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:12.613235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588d0ff-535c-4ce3-aee3-d26395495f9b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:12.618111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:12.619527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:12.619815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:12.620088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] PATH_INFO: `/v3/projects/a7556e22c0c34d548fb49d9dfac50dee` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:12.621277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7556e22c0c34d548fb49d9dfac50dee)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:12.623954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:12.793445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7556e22c0c34d548fb49d9dfac50dee', 'request_id': 'req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:12.892450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7556e22c0c34d548fb49d9dfac50dee', 'request_id': 'req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:12.903346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7556e22c0c34d548fb49d9dfac50dee', 'request_id': 'req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:12.915019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] The token cache is being invalidate because project a7556e22c0c34d548fb49d9dfac50dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:12.915621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:12.916792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:12.918451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 917/1828] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:12 2022] DELETE /identity/v3/projects/a7556e22c0c34d548fb49d9dfac50dee => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:12.923883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:12.942602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4ae7d-e046-4b95-8648-0f66006a5df9 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:12.991298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:13.008610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:13.009240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:13.010045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:13.010435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] PATH_INFO: `/v3/users/02cbd2bd7f914df2a07e2ab3f55dfb7a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:13.011829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cbd2bd7f914df2a07e2ab3f55dfb7a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:13.013853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:13.304684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cbd2bd7f914df2a07e2ab3f55dfb7a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:13.317893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cbd2bd7f914df2a07e2ab3f55dfb7a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:13.326439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cbd2bd7f914df2a07e2ab3f55dfb7a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:13.330486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cbd2bd7f914df2a07e2ab3f55dfb7a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:13.330486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2b99bd84-bb00-47fa-a679-ad45b81e9f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cbd2bd7f914df2a07e2ab3f55dfb7a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:13.591180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 912/1829] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:12 2022] DELETE /identity/v3/users/02cbd2bd7f914df2a07e2ab3f55dfb7a => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:14.105425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:14.106987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:14.131085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c38a4-d113-41ce-b2e2-8b27f15d5fcb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:14.230558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:14.232301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:14.232880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:14.233326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] PATH_INFO: `/v3/projects/ef60bbd1179a4f29823ee8d8b5db9bb1` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:14.234750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef60bbd1179a4f29823ee8d8b5db9bb1)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:14.242834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:14.429479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef60bbd1179a4f29823ee8d8b5db9bb1', 'request_id': 'req-da2655ba-c33b-4e82-8efe-307900505c3d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:14.569147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef60bbd1179a4f29823ee8d8b5db9bb1', 'request_id': 'req-da2655ba-c33b-4e82-8efe-307900505c3d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:14.575846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef60bbd1179a4f29823ee8d8b5db9bb1', 'request_id': 'req-da2655ba-c33b-4e82-8efe-307900505c3d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:14.587365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] The token cache is being invalidate because project ef60bbd1179a4f29823ee8d8b5db9bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:14.587790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:14.589000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:14.590530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 918/1830] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:14 2022] DELETE /identity/v3/projects/ef60bbd1179a4f29823ee8d8b5db9bb1 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:14.596788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c5e3-6efe-40da-914c-3aa7a3727569 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:14.597297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c5e3-6efe-40da-914c-3aa7a3727569 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:14.597391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c5e3-6efe-40da-914c-3aa7a3727569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:14.616923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3ac0c5e3-6efe-40da-914c-3aa7a3727569 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:14.618157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3ac0c5e3-6efe-40da-914c-3aa7a3727569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:14.684754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:14.686669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:14.706026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:14.710720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 913/1831] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:14.719367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:14.719653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:14.720081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:14.752243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:14.754339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:14.764895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 914/1832] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:14.769515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:14.785544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cffc-1bd6-4d75-958f-9cbda8da5a7c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:14.792210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:14.792881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:14.793224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:14.793568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:14.794345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:14.794988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:14.796665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:14.830198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 915/1833] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:14.836419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:14.852566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02779a6-cc8c-441a-a479-9027f4447ee3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:14.865574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:14.866404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:14.866811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:14.867296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:14.868234 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:14.870901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:14.902916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2655ba-c33b-4e82-8efe-307900505c3d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:14.909768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:14.910623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:14.910886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:14.911155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] PATH_INFO: `/v3/users/0650de5d486641f1adc09b40f64b2fa0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:14.912300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0650de5d486641f1adc09b40f64b2fa0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:14.914448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.073057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 916/1834] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:14 2022] POST /identity/v3/projects => generated 371 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:15.079126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:15.104938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:15.159541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c31e7-90cc-43f2-afa5-478c7ab75ad1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:15.167739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:15.168355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.168771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.169053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.169684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:15.171796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.260848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0650de5d486641f1adc09b40f64b2fa0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:15.274210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0650de5d486641f1adc09b40f64b2fa0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:15.282156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0650de5d486641f1adc09b40f64b2fa0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:15.282969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0650de5d486641f1adc09b40f64b2fa0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:15.283620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-221f23ce-bd8c-47ca-9a87-f346217b2c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0650de5d486641f1adc09b40f64b2fa0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:15.306796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 917/1835] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:15 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:15.307977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:15.311082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:15.329304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ed17-8367-4af9-8dd2-7f985431696e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:15.338534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:15.341540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.341540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.341540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.341540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:15.345051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.373269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 918/1836] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:15.377013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:15.388826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1505c72-13ab-4e39-9f4f-c1a07c85928d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:15.393614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:15.394209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.394483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.394857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.395525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:15.397334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.413597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 919/1837] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:15.417063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:15.418472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 919/1838] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:14 2022] DELETE /identity/v3/users/0650de5d486641f1adc09b40f64b2fa0 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:15.424338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.424752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.425286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.438738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8bf1b-0aa1-4ec9-beb0-9c004196ae7f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:15.469449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:15.472493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:15.553121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66ed1f-9b7b-495a-a837-273eb48f743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:15.554335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66ed1f-9b7b-495a-a837-273eb48f743b admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.554622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66ed1f-9b7b-495a-a837-273eb48f743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.555006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66ed1f-9b7b-495a-a837-273eb48f743b admin admin] PATH_INFO: `/v3/projects/369e8ab9aa8742c790d227e781105f11/users/4c86d01c8a794c3c914eaaadce398a6c/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.579225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66ed1f-9b7b-495a-a837-273eb48f743b admin admin] RBAC: Authorizing `identity:create_grant(project_id=369e8ab9aa8742c790d227e781105f11, user_id=4c86d01c8a794c3c914eaaadce398a6c, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:15.583052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66ed1f-9b7b-495a-a837-273eb48f743b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.589416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 920/1839] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:15.615323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:15.616355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:15.680642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97cb02-90da-48a6-b254-7145bc6d4437 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:15.684413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:15.685029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.685698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.686276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.698422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 920/1840] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:15 2022] PUT /identity/v3/projects/369e8ab9aa8742c790d227e781105f11/users/4c86d01c8a794c3c914eaaadce398a6c/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:15.700617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.701068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.701856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.702483 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:15.719106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.725101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:15.726540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:15.727063 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 921/1841] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:15.798097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 921/1842] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:15.807886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:15.809326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:15.820750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe99f1e-e535-4c0c-82a1-ceb31de41952 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:15.870416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:15.871051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:15.874293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:15.874293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:15.880780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:15.882285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:15.891793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 922/1843] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:16.027283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:16.250467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:16.251930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:16.263554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a80ea-4cfa-450b-a53a-ae63f335de51 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:16.268410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:16.269123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:16.269409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:16.269678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] PATH_INFO: `/v3/users/1373fd96ef1e49789ecf827b24c9af4b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:16.270849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1373fd96ef1e49789ecf827b24c9af4b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:16.272820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:16.392264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:16.393863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:16.414126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feac117-199c-45f6-9524-ecc3072b7e84 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:16.440029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:16.440682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:16.440968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:16.441246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] PATH_INFO: `/v3/projects/6757209a9817405894b5a5d960511114` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:16.442453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6757209a9817405894b5a5d960511114)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:16.444548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:16.555515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1373fd96ef1e49789ecf827b24c9af4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.574498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1373fd96ef1e49789ecf827b24c9af4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.583843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1373fd96ef1e49789ecf827b24c9af4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.585297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1373fd96ef1e49789ecf827b24c9af4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.586134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1373fd96ef1e49789ecf827b24c9af4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.643210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6757209a9817405894b5a5d960511114', 'request_id': 'req-952903b7-f80c-40ca-9915-a49d4decd6c7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.700988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 922/1844] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:16 2022] DELETE /identity/v3/users/1373fd96ef1e49789ecf827b24c9af4b => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:16.705739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:16.717705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:16.797121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6757209a9817405894b5a5d960511114', 'request_id': 'req-952903b7-f80c-40ca-9915-a49d4decd6c7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.798225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:16.807639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6757209a9817405894b5a5d960511114', 'request_id': 'req-952903b7-f80c-40ca-9915-a49d4decd6c7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.829341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] The token cache is being invalidate because project 6757209a9817405894b5a5d960511114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:16.829631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.830310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-952903b7-f80c-40ca-9915-a49d4decd6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:16.836148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c056-305a-45cf-9970-447fb35114f9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:16.840365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 923/1845] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:16 2022] DELETE /identity/v3/projects/6757209a9817405894b5a5d960511114 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:16.840365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:16.840365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:16.840365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:16.845683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:16.846868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:16.851648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:16.851648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] PATH_INFO: `/v3/users/8896ffa115fd4da69566177a7c8cadd2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:16.851648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8896ffa115fd4da69566177a7c8cadd2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:16.851648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:16.861752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:16.863025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:16.872458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 924/1846] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:16.874290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:16.875584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:16.893009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d24da-8950-47e3-a1a9-9d0c730129e9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:16.898225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:16.898869 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:16.899151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:16.899425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] PATH_INFO: `/v3/users/a7ece44fb4a547328ff41b1dcac7662a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:16.900658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ece44fb4a547328ff41b1dcac7662a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:16.902951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:17.096045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8896ffa115fd4da69566177a7c8cadd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.108670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8896ffa115fd4da69566177a7c8cadd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.117361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8896ffa115fd4da69566177a7c8cadd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.118377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8896ffa115fd4da69566177a7c8cadd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.119816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8896ffa115fd4da69566177a7c8cadd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.126040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ece44fb4a547328ff41b1dcac7662a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.137800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ece44fb4a547328ff41b1dcac7662a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.145407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ece44fb4a547328ff41b1dcac7662a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.146212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ece44fb4a547328ff41b1dcac7662a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.146854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86e0e893-0cde-414f-a987-2c41ddd0122f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ece44fb4a547328ff41b1dcac7662a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:17.264002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 923/1847] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:16 2022] DELETE /identity/v3/users/8896ffa115fd4da69566177a7c8cadd2 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:17.267542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:17.269053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:17.328286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bff340-3e40-4d3d-938a-b82281a1c976 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:17.331955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:17.332877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:17.332999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:17.333179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:17.334967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:17.378348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 925/1848] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:16 2022] DELETE /identity/v3/users/a7ece44fb4a547328ff41b1dcac7662a => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:17.379891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c93ab-11fb-4866-b8c8-9f07a1463d2e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:17.380169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c93ab-11fb-4866-b8c8-9f07a1463d2e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:17.383427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c93ab-11fb-4866-b8c8-9f07a1463d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:17.402293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7b5c93ab-11fb-4866-b8c8-9f07a1463d2e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:17.403553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7b5c93ab-11fb-4866-b8c8-9f07a1463d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:17.405636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:17.407702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:17.410236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:17.420810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 924/1849] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:17.473516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 926/1850] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:17.490650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:17.493168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:17.504047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e63eec-22c9-478f-af87-401e8e54e430 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:17.524937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:17.565855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fa0e4-1660-4749-b310-459b49975317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:17.566452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa0e4-1660-4749-b310-459b49975317 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:17.566845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa0e4-1660-4749-b310-459b49975317 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:17.567121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa0e4-1660-4749-b310-459b49975317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:17.577587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fa0e4-1660-4749-b310-459b49975317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:17.579143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fa0e4-1660-4749-b310-459b49975317 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:17.598428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 925/1851] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:17.832635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6729-cda1-4cb2-8c48-aecc19a6e2f5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:17.833287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6729-cda1-4cb2-8c48-aecc19a6e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:17.833287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6729-cda1-4cb2-8c48-aecc19a6e2f5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:17.834714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 927/1852] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:14:17.906962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:17.907572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:17.907805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:17.945657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:17.947874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:18.062759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 926/1853] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:14:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:18.170696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:18.170696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:18.171963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:18.207621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] MFA Rules not processed for user `4c86d01c8a794c3c914eaaadce398a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:18.209230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:18.285156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 928/1854] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:18.290093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:18.292069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:18.311747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2e38d-8ec8-4aa7-9f1d-f30d66691f5d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:18.319579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:18.319579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:18.319579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:18.319579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:18.320008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:18.323231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:18.544010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:18.549193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 927/1855] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:18 2022] POST /identity/v3/projects => generated 371 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:18.564008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9d70-5a9b-4981-accb-9766e09f47c9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:18.635355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:18.637497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:18.638081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:18.638726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:18.639135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:18.639409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:18.642243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:18.642243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:18.661083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62767117-9a82-490b-afea-79021e9b4ce1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:18.667772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:18.668572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:18.668694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:18.669724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] PATH_INFO: `/v3/projects/09dd90d18a8348328bf224b3e649d8cf` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:18.671337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09dd90d18a8348328bf224b3e649d8cf)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:18.676505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:18.810229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 929/1856] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:18 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:18.810229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:18.810883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:18.827564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932e97-65cd-4bb6-9f5b-a35637e037c1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:18.834841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:18.836475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:18.836986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:18.837401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] PATH_INFO: `/v3/projects/b52f61cda36f4d8382b4cb8d581dc19a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:18.838638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52f61cda36f4d8382b4cb8d581dc19a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:18.840872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:18.862259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09dd90d18a8348328bf224b3e649d8cf', 'request_id': 'req-87c19ff6-f808-49c7-8094-1c1ead5bad60'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:18.951575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09dd90d18a8348328bf224b3e649d8cf', 'request_id': 'req-87c19ff6-f808-49c7-8094-1c1ead5bad60'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:18.957826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09dd90d18a8348328bf224b3e649d8cf', 'request_id': 'req-87c19ff6-f808-49c7-8094-1c1ead5bad60'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:18.968928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] The token cache is being invalidate because project 09dd90d18a8348328bf224b3e649d8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:18.969221 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:18.969882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:18.972675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 928/1857] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:18 2022] DELETE /identity/v3/projects/09dd90d18a8348328bf224b3e649d8cf => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:18.972675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:18.974430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:18.980757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52f61cda36f4d8382b4cb8d581dc19a', 'request_id': 'req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.038937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c19ff6-f808-49c7-8094-1c1ead5bad60 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.043383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:19.065418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52f61cda36f4d8382b4cb8d581dc19a', 'request_id': 'req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.071726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52f61cda36f4d8382b4cb8d581dc19a', 'request_id': 'req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.084589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] The token cache is being invalidate because project b52f61cda36f4d8382b4cb8d581dc19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:19.084886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.085552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.087308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 930/1858] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:18 2022] DELETE /identity/v3/projects/b52f61cda36f4d8382b4cb8d581dc19a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:19.087844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.089131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.096527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.097158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.097441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.097711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] PATH_INFO: `/v3/projects/3e942fa845c5449f8ae6cd08cbc008fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.099130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e942fa845c5449f8ae6cd08cbc008fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.101026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.140137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:19.152274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd95a7-816e-4957-86ee-6e1f2b9c780e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.156886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.157479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.157751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.158014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.158699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.160268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.170502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 931/1859] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:19.171149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.172733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.217912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e942fa845c5449f8ae6cd08cbc008fb', 'request_id': 'req-e42dd01f-993e-47c7-8b8b-358c31232cca'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.243439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd3990-56a6-48a8-a743-837b4628c979 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.304909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.305546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.305827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.306098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.308559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.311050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e942fa845c5449f8ae6cd08cbc008fb', 'request_id': 'req-e42dd01f-993e-47c7-8b8b-358c31232cca'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.319089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e942fa845c5449f8ae6cd08cbc008fb', 'request_id': 'req-e42dd01f-993e-47c7-8b8b-358c31232cca'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.320762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.322400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfeaae-dd0e-4d22-96ed-3d688e20ec01 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.330842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] The token cache is being invalidate because project 3e942fa845c5449f8ae6cd08cbc008fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:19.331131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.331823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.336245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 932/1860] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:19.336389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 929/1861] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:18 2022] DELETE /identity/v3/projects/3e942fa845c5449f8ae6cd08cbc008fb => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:19.336468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.336468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.336468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.345284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.346961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.355897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:19.357106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.418608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dd01f-993e-47c7-8b8b-358c31232cca admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.432484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 933/1862] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:19.432484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.432484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.432820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.432820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.432820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.432820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] PATH_INFO: `/v3/users/76a07e6b5144468499af1c2be43921f6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.433106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a07e6b5144468499af1c2be43921f6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.438803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.441757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e3c4-7193-4c67-9467-964202cfacb7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.449301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.450421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.450972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.451452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] PATH_INFO: `/v3/projects/93ad5917f4eb42cebe5633e46fef1558/users/18ebe0ccbe664893bfbac3263dfbe11f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.467758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ad5917f4eb42cebe5633e46fef1558, user_id=18ebe0ccbe664893bfbac3263dfbe11f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.470871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.578347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 934/1863] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:19 2022] PUT /identity/v3/projects/93ad5917f4eb42cebe5633e46fef1558/users/18ebe0ccbe664893bfbac3263dfbe11f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:19.583217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.583217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.654094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c977c-d1e3-4775-ad1b-b87421c99777 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.658961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a07e6b5144468499af1c2be43921f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.659087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.659909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.660193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.660470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] PATH_INFO: `/v3/users/5f416f2799bf44b5a61b52cab4a77318` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.661670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f416f2799bf44b5a61b52cab4a77318)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.663791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.673762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a07e6b5144468499af1c2be43921f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.682264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a07e6b5144468499af1c2be43921f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.683287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a07e6b5144468499af1c2be43921f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.684146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4e0eec09-3c57-4be9-b42d-c935a393290d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a07e6b5144468499af1c2be43921f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.769210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:19.791805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 930/1864] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:19 2022] DELETE /identity/v3/users/76a07e6b5144468499af1c2be43921f6 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:19.793126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a262594-9414-42fd-9782-758410bffc20 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.793411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a262594-9414-42fd-9782-758410bffc20 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.793686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a262594-9414-42fd-9782-758410bffc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.815322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9a262594-9414-42fd-9782-758410bffc20 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:19.816617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9a262594-9414-42fd-9782-758410bffc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.877436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 931/1865] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:19.878309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a262594-9414-42fd-9782-758410bffc20 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.881519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9a262594-9414-42fd-9782-758410bffc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.885925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f416f2799bf44b5a61b52cab4a77318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.895899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a262594-9414-42fd-9782-758410bffc20 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.900669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.901326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.901879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.901879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.902524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.904604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.914978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 932/1866] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:19.916357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.921742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f416f2799bf44b5a61b52cab4a77318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.929293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f416f2799bf44b5a61b52cab4a77318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.930214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f416f2799bf44b5a61b52cab4a77318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.930860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f416f2799bf44b5a61b52cab4a77318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:19.931001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042838cd-0234-4228-ac9a-ab3f506302ef admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.938464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.939347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.939668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.940152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.940880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:19.941241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.942932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.955419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 933/1867] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:19.956220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:19.958187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:19.972419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb204d-6de8-45b1-927e-26a4a6db2b84 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:19.977281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:19.978787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:19.979041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:19.979304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] PATH_INFO: `/v3/projects/93ad5917f4eb42cebe5633e46fef1558/users/18ebe0ccbe664893bfbac3263dfbe11f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:19.981582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ad5917f4eb42cebe5633e46fef1558, user_id=18ebe0ccbe664893bfbac3263dfbe11f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:19.984320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:19.996730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 934/1868] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:19 2022] PUT /identity/v3/projects/93ad5917f4eb42cebe5633e46fef1558/users/18ebe0ccbe664893bfbac3263dfbe11f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:20.001137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.008358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:20.041407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 935/1869] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:19 2022] DELETE /identity/v3/users/5f416f2799bf44b5a61b52cab4a77318 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:20.096262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2ae25-9107-4251-9385-9ad6de47bb27 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.175912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.176647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.177495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.177883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.179147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.181352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:20.469821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 935/1870] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:19 2022] POST /identity/v3/projects => generated 365 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:20.469912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.471957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.473662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:20.480870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f699b-f163-4e60-b3bb-a42ba912bf1b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.549536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.550736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.550736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.551118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.551898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.554582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:20.568500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77596e-0bdd-494c-bf43-611f6d6fe9c9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.574142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.575157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.575157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.578442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] PATH_INFO: `/v3/projects/9a789d18b10c4bb4b84b7d9607a67d43` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.578442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a789d18b10c4bb4b84b7d9607a67d43)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.580866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:20.677137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 936/1871] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:20 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:20.683089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.693346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a107c-1b8d-403c-8cbd-f0853b7a0715 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.698338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.703875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.704331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.704799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.710286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.710286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:20.723561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 937/1872] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:20.727389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.728002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a789d18b10c4bb4b84b7d9607a67d43', 'request_id': 'req-73c0b817-f8da-45ae-b99e-0df0272100b4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:20.738158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b7e4-fa79-4dca-9cf7-37a36637804d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.813897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.814516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.814890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.815220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.815915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.817540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:20.821085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a789d18b10c4bb4b84b7d9607a67d43', 'request_id': 'req-73c0b817-f8da-45ae-b99e-0df0272100b4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:20.827822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a789d18b10c4bb4b84b7d9607a67d43', 'request_id': 'req-73c0b817-f8da-45ae-b99e-0df0272100b4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:20.836565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 938/1873] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:20.836565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.841415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:20.847256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:20.864499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69113a7-03d9-4047-9529-29eaab15ad6e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.871381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] The token cache is being invalidate because project 9a789d18b10c4bb4b84b7d9607a67d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:20.871954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:20.872162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.872971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.873251 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.873747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] PATH_INFO: `/v3/projects/ee8eab2601844530bd3cacdc68d249ad` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.875456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8eab2601844530bd3cacdc68d249ad)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.875841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:20.877884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:20.881302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 936/1874] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:20 2022] DELETE /identity/v3/projects/9a789d18b10c4bb4b84b7d9607a67d43 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:20.882229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:20.885199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:20.912761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b817-f8da-45ae-b99e-0df0272100b4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:20.921264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaec7dd-b3e1-42df-b517-f6ff742469a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:20.927060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec7dd-b3e1-42df-b517-f6ff742469a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:20.927060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec7dd-b3e1-42df-b517-f6ff742469a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:20.927060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaec7dd-b3e1-42df-b517-f6ff742469a8 admin admin] PATH_INFO: `/v3/projects/022778f933c1411dbfc24dc9af11453a/users/5a14c403a1694f78993b5191ff67c703/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:20.937731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaec7dd-b3e1-42df-b517-f6ff742469a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=022778f933c1411dbfc24dc9af11453a, user_id=5a14c403a1694f78993b5191ff67c703, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:20.940400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaec7dd-b3e1-42df-b517-f6ff742469a8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.083516 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 937/1875] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:20 2022] PUT /identity/v3/projects/022778f933c1411dbfc24dc9af11453a/users/5a14c403a1694f78993b5191ff67c703/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:21.085995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9fe73-343d-4389-9598-34a3c1067c08 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.086561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9fe73-343d-4389-9598-34a3c1067c08 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.087058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9fe73-343d-4389-9598-34a3c1067c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.091567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8eab2601844530bd3cacdc68d249ad', 'request_id': 'req-71bd7c88-0e30-451d-b8bc-183a04e3db4e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:21.111851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-51a9fe73-343d-4389-9598-34a3c1067c08 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:21.113525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-51a9fe73-343d-4389-9598-34a3c1067c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.173913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 938/1876] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:21.180895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.180895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.180895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.197854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8eab2601844530bd3cacdc68d249ad', 'request_id': 'req-71bd7c88-0e30-451d-b8bc-183a04e3db4e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:21.206496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8eab2601844530bd3cacdc68d249ad', 'request_id': 'req-71bd7c88-0e30-451d-b8bc-183a04e3db4e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:21.209457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] MFA Rules not processed for user `5a14c403a1694f78993b5191ff67c703`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:21.210997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.222454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] The token cache is being invalidate because project ee8eab2601844530bd3cacdc68d249ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:21.222864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:21.223866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-71bd7c88-0e30-451d-b8bc-183a04e3db4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:21.229471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 939/1877] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:20 2022] DELETE /identity/v3/projects/ee8eab2601844530bd3cacdc68d249ad => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:21.229471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5fc5-9b6a-4c35-93cb-9d1e1f0fc1a2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.229471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5fc5-9b6a-4c35-93cb-9d1e1f0fc1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.229471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5fc5-9b6a-4c35-93cb-9d1e1f0fc1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.255481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-96ae5fc5-9b6a-4c35-93cb-9d1e1f0fc1a2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:21.256935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-96ae5fc5-9b6a-4c35-93cb-9d1e1f0fc1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.267312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 940/1878] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:21.268905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.269314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.269641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.294543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 939/1879] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:21.294789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:21.294895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.294986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.317309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c258e0-e844-4ebe-ace3-3094117d4ab3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.317539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 941/1880] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:21.317539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.317539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.318983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.318983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.319467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.319994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.323251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:21.323948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.326428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b808eb1-7ac2-47ad-a806-fd0aed884474 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.333439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3549636-3fbf-48b3-8705-db7566d7053c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.340300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.340985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.342539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.342649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 940/1881] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:21.342769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.342769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.343432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.343786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.344125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.345209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.363950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:21.371219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:21.371219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.377261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 941/1882] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:21.378022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.388189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543cb7e-ac71-4b29-9434-f190e7e0818c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.395571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.396189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.396469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.396748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.398086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.400071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.532463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 942/1883] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/projects => generated 363 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:21.532463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.535077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 942/1884] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:21.535954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.613005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834fff7-5db5-4c6a-abf1-3e83c3c5d857 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.616725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7da2-ae14-41c6-bdb2-836cf2bea2e3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.618096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.619378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.620658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.620658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.621474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:21.621474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.623155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.635734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.636624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.636977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.640609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.640762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 943/1885] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:21.640848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.640923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.640996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.652414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6049-0237-4383-b2b1-4e8d077982b1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.657143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83254e1a-7467-4aba-8100-0e9cb1c50477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.657788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83254e1a-7467-4aba-8100-0e9cb1c50477 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.658062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83254e1a-7467-4aba-8100-0e9cb1c50477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.658452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83254e1a-7467-4aba-8100-0e9cb1c50477 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.659117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83254e1a-7467-4aba-8100-0e9cb1c50477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.661310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83254e1a-7467-4aba-8100-0e9cb1c50477 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.746606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 943/1886] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:21.751884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.757504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08218-1b4e-42d7-adde-cf6f59ad5a24 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:21.763581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:21.764180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.764464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.764765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.765391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:21.767425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:21.827368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 944/1887] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:21.828978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:21.829149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:21.829727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:21.859879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] MFA Rules not processed for user `18ebe0ccbe664893bfbac3263dfbe11f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:21.861250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:21.909141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 944/1888] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/projects => generated 393 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:21.909141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.942824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 945/1889] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:21.948728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:21.965073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93845ae-4617-407e-8890-bc9a558d3466 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.004792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff7c1-c52b-40d0-88f6-b53339f2fdca admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.016587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.017981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.017981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.018388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.019155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.021922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.032045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.032682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.033013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.033288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.035089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.038439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.041311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 945/1890] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:22.044656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.053520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361e5ec-f65e-4623-8c04-37474fe1be8a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.057375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 946/1891] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:22.057375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.057375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:22.062797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.063537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.063919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.064504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.066580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.069779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.120279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd9ce-10e4-44c3-8155-4376e869ac42 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.124054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.124734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.125018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.125290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.127148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:22.131233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:22.167336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 946/1892] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:22 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:22.167336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.175536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aab48-c44a-4bf5-8b9c-b2c5d46f5043 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.182442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.183143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.183414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.183731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] PATH_INFO: `/v3/projects/13e86665f4a4402da0f0d3e6068026ce/users/cfa4b07effa346f0acdde163db1622d3/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.196146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.197602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.199790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e86665f4a4402da0f0d3e6068026ce, user_id=cfa4b07effa346f0acdde163db1622d3, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.202402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.207773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 947/1893] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:22.208889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.220655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8125-7cfb-44bc-b556-42933013ca4c service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.234490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.235155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.235437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.235854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.236502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.244427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.258829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 948/1894] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:22.260646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.276709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f612b-5292-4925-8ea4-448c4ea58dd3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.284479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.285099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.286118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.286252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.286935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.288547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.292110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 947/1895] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:22 2022] PUT /identity/v3/projects/13e86665f4a4402da0f0d3e6068026ce/users/cfa4b07effa346f0acdde163db1622d3/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:22.292110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.301385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 949/1896] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:22.303902 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.316081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c7f4e-1e84-486d-991d-a365b92b21c9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.383786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b597a-0892-4dc8-a75e-b2b816504f43 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.389707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ebe8b-d8ff-4b17-bd4d-5a48b874e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.390471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ebe8b-d8ff-4b17-bd4d-5a48b874e114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.390801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ebe8b-d8ff-4b17-bd4d-5a48b874e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.390948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.391079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ebe8b-d8ff-4b17-bd4d-5a48b874e114 admin admin] PATH_INFO: `/v3/projects/ea8b967121374612b9e657e3fc5ba9d3/users/335966c8131b459a8fd4851a0c146154/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.391530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.391875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.392153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.393679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.394464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.405376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 950/1897] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:22.406330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.407326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ebe8b-d8ff-4b17-bd4d-5a48b874e114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8b967121374612b9e657e3fc5ba9d3, user_id=335966c8131b459a8fd4851a0c146154, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.410649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ebe8b-d8ff-4b17-bd4d-5a48b874e114 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.417163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad96d-d24a-48b6-84a8-29b9186eae18 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.422492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.423103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.425523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.425523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.425523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.425987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.439448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 951/1898] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:22.444601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.452991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252913a-21d0-4374-9392-457f08d53af3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.457521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.458522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.458522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.458770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] PATH_INFO: `/v3/projects/13e86665f4a4402da0f0d3e6068026ce/users/cfa4b07effa346f0acdde163db1622d3/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.462691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e86665f4a4402da0f0d3e6068026ce, user_id=cfa4b07effa346f0acdde163db1622d3, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.463597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.479884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 952/1899] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:22 2022] PUT /identity/v3/projects/13e86665f4a4402da0f0d3e6068026ce/users/cfa4b07effa346f0acdde163db1622d3/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:22.484198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.515181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 948/1900] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:22 2022] PUT /identity/v3/projects/ea8b967121374612b9e657e3fc5ba9d3/users/335966c8131b459a8fd4851a0c146154/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:22.516635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e994fb0-b365-4ce2-81e2-b7ecd63d9d53 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.518803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e994fb0-b365-4ce2-81e2-b7ecd63d9d53 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.518803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e994fb0-b365-4ce2-81e2-b7ecd63d9d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.547360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1e994fb0-b365-4ce2-81e2-b7ecd63d9d53 None None] MFA Rules not processed for user `cfa4b07effa346f0acdde163db1622d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:22.548614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1e994fb0-b365-4ce2-81e2-b7ecd63d9d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:22.563141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab934602-dd1f-426f-80a5-a0c9c157b15e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.644508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.646390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.646723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.647009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] PATH_INFO: `/v3/projects/5928ebd8c424442eb2bf9eaf0e7590eb/users/bcb7e2107b0a436fa8a1e938a9293384/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.655501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 949/1901] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:22.657148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c9dfe-4ccd-4fb7-8ae5-855fdfa20044 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.657771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c9dfe-4ccd-4fb7-8ae5-855fdfa20044 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.658245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c9dfe-4ccd-4fb7-8ae5-855fdfa20044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.678569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:22.683633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5928ebd8c424442eb2bf9eaf0e7590eb, user_id=bcb7e2107b0a436fa8a1e938a9293384, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.686335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.708758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-935c9dfe-4ccd-4fb7-8ae5-855fdfa20044 None None] MFA Rules not processed for user `335966c8131b459a8fd4851a0c146154`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:22.710772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-935c9dfe-4ccd-4fb7-8ae5-855fdfa20044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:22.758390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 953/1902] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:22 2022] PUT /identity/v3/projects/5928ebd8c424442eb2bf9eaf0e7590eb/users/bcb7e2107b0a436fa8a1e938a9293384/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:22.760054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.761702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:22.814374 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 950/1903] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:22.819242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.819530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.819809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.828711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeab03-d901-415c-968c-10e51123941a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.833690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.834297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.834577 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.834879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.848124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] MFA Rules not processed for user `bcb7e2107b0a436fa8a1e938a9293384`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:22.849351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:22.907468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.909607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:22.921644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 954/1904] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:22.922529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.934991 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 951/1905] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:22.942266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:22.990638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171ca9-5bff-4113-8d93-c5476f0e9ea4 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:22.995968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:22.996571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:22.996853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:22.997126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:22.997743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:22.999700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.013418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac75cd1-dc90-4bad-b88e-ad7ce00e6956 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.018017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.018608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.018918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.019217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.019810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.025044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.119649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 955/1906] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:22 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:23.123501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.200266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 952/1907] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:22 2022] POST /identity/v3/projects => generated 393 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:23.202182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.205671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67877e-e47b-4822-80cd-424b1a08d055 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.244671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:23.298985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0e74c-411c-48b6-af3a-84d6e059e78c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.304930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.305202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.305582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.305954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.306987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.309779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.370578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.371387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.371730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.372251 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.372982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.375699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.524376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 956/1908] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:23 2022] POST /identity/v3/users => generated 298 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:23.526118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.533137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 953/1909] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:23 2022] POST /identity/v3/users => generated 303 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:23.534552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.558089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2b811-be83-46e4-aa88-4bb3daae1fe4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.573183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.573818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.574108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.574363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.575043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.577258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.604414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 954/1910] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:23.605902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.619266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b85e9-e622-4561-a9bb-c45ab964de44 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.624427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.625164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.625538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.625897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.626638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.628372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.643289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 955/1911] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:23.644092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.647726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93416d-81cf-4443-a81f-a4e1a8534e6e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.653228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.655788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.655788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.655788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.663994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daf61d-0758-4e75-9204-55e710fbfc28 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.672447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.673446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.674201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.674474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] PATH_INFO: `/v3/projects/484150dc1dcb413aa93fdd013f3551f3/users/1b5b4a4ad8fa4ba0873dd4eb4834ac23/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.700989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] RBAC: Authorizing `identity:create_grant(project_id=484150dc1dcb413aa93fdd013f3551f3, user_id=1b5b4a4ad8fa4ba0873dd4eb4834ac23, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.705414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.769555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.771607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.785620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 957/1912] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:23.785844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.800331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e6dcc-a1c9-4f80-aeb0-03eb8a77e043 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.814862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 956/1913] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:23 2022] PUT /identity/v3/projects/484150dc1dcb413aa93fdd013f3551f3/users/1b5b4a4ad8fa4ba0873dd4eb4834ac23/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:23.816044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.827989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.828694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.828966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.829497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] PATH_INFO: `/v3/projects/fce095661af842779bbb0efe087f5f3f/users/0bf02677063740fd89f24d8de236be84/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.871328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce095661af842779bbb0efe087f5f3f, user_id=0bf02677063740fd89f24d8de236be84, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.873414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.918345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b68a-681b-4cad-8f79-6e59fc07214e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.923617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.924461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.924921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.925343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.926128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.928153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:23.940010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 957/1914] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:23.948322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:23.960183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23cbcd-30e6-408d-acc6-8181bf438875 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:23.967041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1ff42-7ba7-4fc7-bceb-c4f4b6181980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:23.967877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1ff42-7ba7-4fc7-bceb-c4f4b6181980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:23.968484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1ff42-7ba7-4fc7-bceb-c4f4b6181980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:23.968781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1ff42-7ba7-4fc7-bceb-c4f4b6181980 admin admin] PATH_INFO: `/v3/projects/484150dc1dcb413aa93fdd013f3551f3/users/1b5b4a4ad8fa4ba0873dd4eb4834ac23/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:23.972076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1ff42-7ba7-4fc7-bceb-c4f4b6181980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484150dc1dcb413aa93fdd013f3551f3, user_id=1b5b4a4ad8fa4ba0873dd4eb4834ac23, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:23.975203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1ff42-7ba7-4fc7-bceb-c4f4b6181980 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.000430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 958/1915] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:23 2022] PUT /identity/v3/projects/484150dc1dcb413aa93fdd013f3551f3/users/1b5b4a4ad8fa4ba0873dd4eb4834ac23/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:24.002550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.002881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.003155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.014330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 958/1916] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:23 2022] PUT /identity/v3/projects/fce095661af842779bbb0efe087f5f3f/users/0bf02677063740fd89f24d8de236be84/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:24.015415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:24.035573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] MFA Rules not processed for user `1b5b4a4ad8fa4ba0873dd4eb4834ac23`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:24.036618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:24.131510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1624ed-c1f3-46c9-bcfe-fca0bae5c9ed admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:24.137442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:24.140139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.140139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.140139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.140139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:24.140974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.145971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 959/1917] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:24.155645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:24.168152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 959/1918] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:24.169863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:24.181052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b29e4-66ed-45db-b1a2-693547f94ecc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:24.186517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a405-61e7-4cc4-a616-e9295011ae6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:24.187230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a405-61e7-4cc4-a616-e9295011ae6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.187508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a405-61e7-4cc4-a616-e9295011ae6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.187780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a405-61e7-4cc4-a616-e9295011ae6a admin admin] PATH_INFO: `/v3/projects/fce095661af842779bbb0efe087f5f3f/users/0bf02677063740fd89f24d8de236be84/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.189663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414a405-61e7-4cc4-a616-e9295011ae6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce095661af842779bbb0efe087f5f3f, user_id=0bf02677063740fd89f24d8de236be84, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:24.192584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414a405-61e7-4cc4-a616-e9295011ae6a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.204047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 960/1919] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:24 2022] PUT /identity/v3/projects/fce095661af842779bbb0efe087f5f3f/users/0bf02677063740fd89f24d8de236be84/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:24.215131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.215418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.215697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.220878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776ba49-6031-45e3-a148-a11b601f108c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:24.252916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] MFA Rules not processed for user `0bf02677063740fd89f24d8de236be84`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:24.252916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:24.270333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:24.301563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:24.302253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.302826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.303119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.336110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 961/1920] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:24.346829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:24.356374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76fff-284e-4062-b74c-023d2a72354a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:24.362951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:24.363715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.364003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.364277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.364538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:24.366094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.374562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:24.376027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.376451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 960/1921] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:24.389703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 962/1922] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:24 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:24.446952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:24.456166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ac04-3747-4256-b01e-774729acd647 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:24.461380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:24.461989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.462268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.462590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.489235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:24.498505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d719e0-0cec-4c9d-82a8-9c28d95089c9 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:24.502916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:24.503447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:24.503791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:24.504157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:24.521352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:24.522831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.531613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 961/1923] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:24.560710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:24.562143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:24.571355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 963/1924] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:25.486444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:25.495137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73970ded-a112-4ef2-93e6-6b9ed37efe4e service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:25.498808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebca564-49e3-4367-8d89-99f8b2835570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:25.506533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebca564-49e3-4367-8d89-99f8b2835570 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:25.506533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebca564-49e3-4367-8d89-99f8b2835570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:25.506533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebca564-49e3-4367-8d89-99f8b2835570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:25.562048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebca564-49e3-4367-8d89-99f8b2835570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:25.563560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebca564-49e3-4367-8d89-99f8b2835570 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:25.574424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 962/1925] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:25.950165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:26.972667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:28.282630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:28.393488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca4402-a742-4dc4-b025-dd176a507373 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:28.398313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:28.399171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:28.399371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:28.399627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] PATH_INFO: `/v3/users/5a14c403a1694f78993b5191ff67c703` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:28.401520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a14c403a1694f78993b5191ff67c703)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:28.406785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:29.066440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a14c403a1694f78993b5191ff67c703'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:29.084748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a14c403a1694f78993b5191ff67c703'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:29.113872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a14c403a1694f78993b5191ff67c703'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:29.114932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a14c403a1694f78993b5191ff67c703'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:29.115766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a14c403a1694f78993b5191ff67c703'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:29.233325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:29.282089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 964/1926] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:28 2022] DELETE /identity/v3/users/5a14c403a1694f78993b5191ff67c703 => generated 0 bytes in 998 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:29.284242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:29.284801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:29.284801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:29.304836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:29.306098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:29.376056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 963/1927] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:29.386224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:29.387683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:29.415153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:29.469699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07469a63-3a86-4b2f-bcc9-0cdf7d89c49d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:29.473881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:29.474505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:29.474852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:29.475139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:29.486719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:29.488518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:29.499848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 965/1928] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:29.839589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:30.358194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:30.360346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:30.374238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed5309-a2ea-4bef-a717-3a2f10386718 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:30.379246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:30.380822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:30.381261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:30.381691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] PATH_INFO: `/v3/projects/022778f933c1411dbfc24dc9af11453a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:30.383038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=022778f933c1411dbfc24dc9af11453a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:30.385325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:30.390792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:30.392496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:30.419284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf452-6430-406f-9470-83e7c1574179 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:30.424129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:30.425062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:30.425442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:30.425822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] PATH_INFO: `/v3/users/335966c8131b459a8fd4851a0c146154` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:30.427097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335966c8131b459a8fd4851a0c146154)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:30.429685 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:30.572910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022778f933c1411dbfc24dc9af11453a', 'request_id': 'req-98068051-14b7-4104-83ee-381c655e9e86'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.849104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022778f933c1411dbfc24dc9af11453a', 'request_id': 'req-98068051-14b7-4104-83ee-381c655e9e86'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.858263 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022778f933c1411dbfc24dc9af11453a', 'request_id': 'req-98068051-14b7-4104-83ee-381c655e9e86'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.863690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335966c8131b459a8fd4851a0c146154'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.871015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] The token cache is being invalidate because project 022778f933c1411dbfc24dc9af11453a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:30.871894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.872696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.874336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 964/1929] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:30 2022] DELETE /identity/v3/projects/022778f933c1411dbfc24dc9af11453a => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:30.876300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335966c8131b459a8fd4851a0c146154'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.878249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:30.879608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:30.883659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335966c8131b459a8fd4851a0c146154'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.884711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335966c8131b459a8fd4851a0c146154'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.885510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-34b1958f-f80d-4a5c-902e-9607b2da2ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335966c8131b459a8fd4851a0c146154'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:30.895928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:30.946850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98068051-14b7-4104-83ee-381c655e9e86 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:30.951590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:30.952229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:30.952499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:30.952768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] PATH_INFO: `/v3/users/cfa4b07effa346f0acdde163db1622d3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:30.953891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa4b07effa346f0acdde163db1622d3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:30.955867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:31.019740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 966/1930] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:30 2022] DELETE /identity/v3/users/335966c8131b459a8fd4851a0c146154 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:31.024186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af68c49-4a68-4748-9337-58956e19950d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:31.024526 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af68c49-4a68-4748-9337-58956e19950d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:31.024856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af68c49-4a68-4748-9337-58956e19950d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:31.044059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7af68c49-4a68-4748-9337-58956e19950d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:31.045253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7af68c49-4a68-4748-9337-58956e19950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:31.104974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 967/1931] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:31.114704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af68c49-4a68-4748-9337-58956e19950d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:31.116042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7af68c49-4a68-4748-9337-58956e19950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:31.178600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af68c49-4a68-4748-9337-58956e19950d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:31.182802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:31.185268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:31.185268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:31.185268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:31.194734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:31.197091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:31.207871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 968/1932] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:31.251812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa4b07effa346f0acdde163db1622d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:31.267196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa4b07effa346f0acdde163db1622d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:31.275465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa4b07effa346f0acdde163db1622d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:31.276339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa4b07effa346f0acdde163db1622d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:31.277015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa4b07effa346f0acdde163db1622d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:31.479767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 965/1933] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:30 2022] DELETE /identity/v3/users/cfa4b07effa346f0acdde163db1622d3 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:31.650410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:31.775711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:32.392588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:32.394039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:32.408669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92ce85-99b8-4661-9e40-87398b5ce0d5 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:32.527802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:32.528381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:32.528656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:32.528927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] PATH_INFO: `/v3/users/bcb7e2107b0a436fa8a1e938a9293384` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:32.530138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb7e2107b0a436fa8a1e938a9293384)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:32.532257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:32.661508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:32.663279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:32.699803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6e8de-db13-4eea-8c4c-9d21c26bcdfb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:32.710266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:32.713450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:32.713450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:32.713450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] PATH_INFO: `/v3/projects/ea8b967121374612b9e657e3fc5ba9d3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:32.714196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8b967121374612b9e657e3fc5ba9d3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:32.716215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:32.798623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:32.808317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb7e2107b0a436fa8a1e938a9293384'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:32.820632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb7e2107b0a436fa8a1e938a9293384'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:32.828533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb7e2107b0a436fa8a1e938a9293384'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:32.829366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb7e2107b0a436fa8a1e938a9293384'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:32.830029 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb7e2107b0a436fa8a1e938a9293384'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:32.883299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8b967121374612b9e657e3fc5ba9d3', 'request_id': 'req-3fa5681b-84da-4746-b1b7-75e1cb1b367a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:32.963940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 969/1934] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:32 2022] DELETE /identity/v3/users/bcb7e2107b0a436fa8a1e938a9293384 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:32.965396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:32.969733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:32.997386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:33.037970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960b067-d2f2-4220-811a-0057497696dd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:33.044125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:33.044768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.045043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.045316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] PATH_INFO: `/v3/projects/13e86665f4a4402da0f0d3e6068026ce` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.046717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e86665f4a4402da0f0d3e6068026ce)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:33.048727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:33.054213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8b967121374612b9e657e3fc5ba9d3', 'request_id': 'req-3fa5681b-84da-4746-b1b7-75e1cb1b367a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.063996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8b967121374612b9e657e3fc5ba9d3', 'request_id': 'req-3fa5681b-84da-4746-b1b7-75e1cb1b367a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.079288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] The token cache is being invalidate because project ea8b967121374612b9e657e3fc5ba9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:33.079820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.081022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fa5681b-84da-4746-b1b7-75e1cb1b367a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.087882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 966/1935] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:32 2022] DELETE /identity/v3/projects/ea8b967121374612b9e657e3fc5ba9d3 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:33.089305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256028c3-da4b-400f-9245-aca305c1911d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.089872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256028c3-da4b-400f-9245-aca305c1911d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.090600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256028c3-da4b-400f-9245-aca305c1911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.117281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-256028c3-da4b-400f-9245-aca305c1911d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:33.119781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-256028c3-da4b-400f-9245-aca305c1911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:33.131197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 967/1936] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:33.132160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256028c3-da4b-400f-9245-aca305c1911d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:33.137164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-256028c3-da4b-400f-9245-aca305c1911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:33.152847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256028c3-da4b-400f-9245-aca305c1911d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:33.157648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:33.158342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.159255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.159529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] PATH_INFO: `/v3/users/1b5b4a4ad8fa4ba0873dd4eb4834ac23` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.161352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b5b4a4ad8fa4ba0873dd4eb4834ac23)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:33.163311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:33.203888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e86665f4a4402da0f0d3e6068026ce', 'request_id': 'req-edf1239b-725a-48aa-bfae-8c4d5447bfa1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.414796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e86665f4a4402da0f0d3e6068026ce', 'request_id': 'req-edf1239b-725a-48aa-bfae-8c4d5447bfa1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.421537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e86665f4a4402da0f0d3e6068026ce', 'request_id': 'req-edf1239b-725a-48aa-bfae-8c4d5447bfa1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.433663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] The token cache is being invalidate because project 13e86665f4a4402da0f0d3e6068026ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:33.434201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.435645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.437148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 970/1937] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:32 2022] DELETE /identity/v3/projects/13e86665f4a4402da0f0d3e6068026ce => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:33.437911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:33.439239 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:33.500933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1239b-725a-48aa-bfae-8c4d5447bfa1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:33.504645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:33.505473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.506147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.506626 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.508490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:33.582728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:33.583942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f1c08-6bc1-43d8-ba72-18abbef55cd7 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:33.593783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 971/1938] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:33.595130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe828aa-397b-43b4-a53f-07c1be5f128c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.595652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe828aa-397b-43b4-a53f-07c1be5f128c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.595924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe828aa-397b-43b4-a53f-07c1be5f128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.622102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5fe828aa-397b-43b4-a53f-07c1be5f128c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:33.624125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5fe828aa-397b-43b4-a53f-07c1be5f128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:33.630227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b5b4a4ad8fa4ba0873dd4eb4834ac23'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.637942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 972/1939] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:33.643804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b5b4a4ad8fa4ba0873dd4eb4834ac23'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.647475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.647773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.648048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.667912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b5b4a4ad8fa4ba0873dd4eb4834ac23'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.668662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b5b4a4ad8fa4ba0873dd4eb4834ac23'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.669351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b5b4a4ad8fa4ba0873dd4eb4834ac23'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:33.675134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:33.681842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:33.686706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 973/1940] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:33.690722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:33.710559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfdced-ae0c-4b29-a372-f9940801570e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:33.710559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:33.710855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.711007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.711274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.711945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:33.712762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:33.714356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:33.726819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 974/1941] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:33.731471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:33.741918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf1dcf-5796-410f-8332-b85fb60d02b6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:33.749168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:33.749955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:33.751170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:33.751670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:33.755563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:33.762266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:33.829430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 968/1942] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:33 2022] DELETE /identity/v3/users/1b5b4a4ad8fa4ba0873dd4eb4834ac23 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:33.931446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 975/1943] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:33 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:33.935613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.022225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4330d-d7a6-4ba1-a1ff-947896c8e268 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:34.177458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:34.178087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.178370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.178646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.179309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:34.182697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:34.296513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:34.303425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 969/1944] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:33 2022] POST /identity/v3/users => generated 290 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:34.307765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.318719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9829fdf-6bb8-4399-8b9f-9f7ddcc7dcda admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:34.324638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:34.325376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.325727 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.326249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.327054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:34.329236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:34.387998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 976/1945] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:34.390568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.410524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c753e-dc1e-4a5d-8415-8cecb8ceff78 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:34.415933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:34.416649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.416992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.417353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.418681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:34.420491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:34.441662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 970/1946] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:34.445858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.457098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5462a-c1e0-4e77-8fcf-251f0dac6346 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:34.463226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ca3d8-f7af-465d-a2ac-600c65fa5267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:34.463896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ca3d8-f7af-465d-a2ac-600c65fa5267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.464183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ca3d8-f7af-465d-a2ac-600c65fa5267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.464183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ca3d8-f7af-465d-a2ac-600c65fa5267 admin admin] PATH_INFO: `/v3/projects/857c6684f3c74c14a781cee43615d0cb/users/727502d004564f1cb0d75a5e02720622/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.493020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ca3d8-f7af-465d-a2ac-600c65fa5267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=857c6684f3c74c14a781cee43615d0cb, user_id=727502d004564f1cb0d75a5e02720622, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:34.496641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ca3d8-f7af-465d-a2ac-600c65fa5267 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:34.536590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.538379 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:34.554087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a5d8f-f1ce-4c07-91c0-86b03be6da76 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:34.562319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:34.563545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.563839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.564108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] PATH_INFO: `/v3/projects/5928ebd8c424442eb2bf9eaf0e7590eb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.572514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5928ebd8c424442eb2bf9eaf0e7590eb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:34.572514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:34.654108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 977/1947] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:34 2022] PUT /identity/v3/projects/857c6684f3c74c14a781cee43615d0cb/users/727502d004564f1cb0d75a5e02720622/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:34.664747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.665058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.665499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.704665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] MFA Rules not processed for user `727502d004564f1cb0d75a5e02720622`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:34.704665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:34.810649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 978/1948] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:34.810845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5928ebd8c424442eb2bf9eaf0e7590eb', 'request_id': 'req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:34.812894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.902105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfb0ae-b65a-4198-b234-8ac897c220fe None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:34.917762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5928ebd8c424442eb2bf9eaf0e7590eb', 'request_id': 'req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:34.924602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5928ebd8c424442eb2bf9eaf0e7590eb', 'request_id': 'req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:34.945084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] The token cache is being invalidate because project 5928ebd8c424442eb2bf9eaf0e7590eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:34.945084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:34.945084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:34.951590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 971/1949] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:34 2022] DELETE /identity/v3/projects/5928ebd8c424442eb2bf9eaf0e7590eb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:34.953690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:34.956009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:34.977392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:34.979492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:34.980083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:34.980552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:34.981583 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:34.984077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:34.990307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:35.043242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdec7e9-0728-4ee6-b9b2-8c533cb732d1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.048961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.049664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.050005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.050561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] PATH_INFO: `/v3/projects/484150dc1dcb413aa93fdd013f3551f3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.051858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=484150dc1dcb413aa93fdd013f3551f3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.055763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.092513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 979/1950] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:34 2022] POST /identity/v3/projects => generated 381 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:35.094078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.102221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:35.180919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c83f-5ac6-43e4-96c7-6c11bda5f0eb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.185827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.186730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.186980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.187249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] PATH_INFO: `/v3/users/0bf02677063740fd89f24d8de236be84` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.188366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf02677063740fd89f24d8de236be84)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.190442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.221944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484150dc1dcb413aa93fdd013f3551f3', 'request_id': 'req-1ed23cde-cf98-4710-9c56-249bfab16ba3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.334391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484150dc1dcb413aa93fdd013f3551f3', 'request_id': 'req-1ed23cde-cf98-4710-9c56-249bfab16ba3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.341278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484150dc1dcb413aa93fdd013f3551f3', 'request_id': 'req-1ed23cde-cf98-4710-9c56-249bfab16ba3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.353671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] The token cache is being invalidate because project 484150dc1dcb413aa93fdd013f3551f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:35.353989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.355088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.356547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 972/1951] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:34 2022] DELETE /identity/v3/projects/484150dc1dcb413aa93fdd013f3551f3 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:35.357586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.359166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:35.426938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed23cde-cf98-4710-9c56-249bfab16ba3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.433486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b2783-41b7-45b2-a0db-74417a219c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.434117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2783-41b7-45b2-a0db-74417a219c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.434572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2783-41b7-45b2-a0db-74417a219c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.435209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2783-41b7-45b2-a0db-74417a219c75 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.436171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b2783-41b7-45b2-a0db-74417a219c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.438228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b2783-41b7-45b2-a0db-74417a219c75 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.439627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf02677063740fd89f24d8de236be84'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.452134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf02677063740fd89f24d8de236be84'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.460168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf02677063740fd89f24d8de236be84'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.460928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf02677063740fd89f24d8de236be84'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.461561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f17358b-625b-49f4-a164-abae935fb4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf02677063740fd89f24d8de236be84'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:35.512877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 973/1952] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:35 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:35.513942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.514223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.514495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.534774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:35.536012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:35.543970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 974/1953] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:35.544741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.546118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:35.562643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 980/1954] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:35 2022] DELETE /identity/v3/users/0bf02677063740fd89f24d8de236be84 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:35.562643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.562643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.562643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.592014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb6579-8b80-4aa3-b29f-9ead326cc047 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.609211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:35.611080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:35.688499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 981/1955] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:35.693020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.702190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.704041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.704416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ac74f-e4ce-48b2-9732-b404661828d8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.709367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.709367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.709367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.710594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.716439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.717057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.717335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.717607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.718413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:35.718747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.720704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.738689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 975/1956] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:35.743064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.743976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 982/1957] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:35.745203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.759766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bfad7-61cd-400a-b563-325e71a8bdff admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.769762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc72e-6a56-46dc-a9d5-bdda95a150e8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:35.773171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.774056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.774729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.776622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.777494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.778082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:35.780099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.780997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:35.781300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:35.781580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] PATH_INFO: `/v3/projects/b34e1488736b464ca361d405d2ea58f9/users/934a08bc45bf468880fce284b5261d8b/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:35.804385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34e1488736b464ca361d405d2ea58f9, user_id=934a08bc45bf468880fce284b5261d8b, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:35.808518 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:35.910123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 983/1958] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:35 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:35.916525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:35.927558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:35.934533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 976/1959] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:35 2022] PUT /identity/v3/projects/b34e1488736b464ca361d405d2ea58f9/users/934a08bc45bf468880fce284b5261d8b/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:35.938874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.013804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbf889-9fa7-444c-8b35-e747848c6973 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.014363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6afef6-dc5e-4b93-9d40-2f71c01f8107 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.022254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69af26-0058-4ce7-bcd5-eefa0fc3f536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.023178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69af26-0058-4ce7-bcd5-eefa0fc3f536 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.023550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69af26-0058-4ce7-bcd5-eefa0fc3f536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.024402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69af26-0058-4ce7-bcd5-eefa0fc3f536 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.025205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69af26-0058-4ce7-bcd5-eefa0fc3f536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.027355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69af26-0058-4ce7-bcd5-eefa0fc3f536 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.034643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.035308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.035589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.035863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.042376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.044583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.056847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 977/1960] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:36.061083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.071019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067765d1-ef46-4909-9708-648de45f6869 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.076503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.077365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.077642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.077914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] PATH_INFO: `/v3/projects/b34e1488736b464ca361d405d2ea58f9/users/934a08bc45bf468880fce284b5261d8b/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.081394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34e1488736b464ca361d405d2ea58f9, user_id=934a08bc45bf468880fce284b5261d8b, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.082749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.101031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 984/1962] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:35 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:36.102905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 978/1962] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:36 2022] PUT /identity/v3/projects/b34e1488736b464ca361d405d2ea58f9/users/934a08bc45bf468880fce284b5261d8b/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:36.105841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.106799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.107467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.107592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.147247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] MFA Rules not processed for user `934a08bc45bf468880fce284b5261d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:36.149169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.203338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a656a2b-63b0-410b-ab77-3bcc08d184dc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.208530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.209282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.209684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.209964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.210673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.212576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.223935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 979/1963] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:36.228089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.240436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 985/1964] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:36.242266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458247b-40f8-433f-98a9-c3b6253c9ea3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.246228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:36.251062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.251062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.251062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.251389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.252330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.255477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.264097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.268452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.296717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 980/1965] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:36.297748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.302738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.320878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09705a-bf3d-47c6-8e11-3a965ff73171 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.331135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.331135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.331135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.331135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] PATH_INFO: `/v3/projects/fce095661af842779bbb0efe087f5f3f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.331495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce095661af842779bbb0efe087f5f3f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.333119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.373078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481176e-4505-468a-873f-78a43baae5e6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.376749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.377360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.381657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.381657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.387159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.388581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.399571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 986/1966] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:36.400413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.411080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cb6e7-a4c4-4d3b-bd3e-b5872d7a6688 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.416716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988208b2-3b8b-451c-9ed3-70acb3d2e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.417371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988208b2-3b8b-451c-9ed3-70acb3d2e75a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.417957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988208b2-3b8b-451c-9ed3-70acb3d2e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.417957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988208b2-3b8b-451c-9ed3-70acb3d2e75a admin admin] PATH_INFO: `/v3/projects/863745b4145f48a6b00a4e669753565e/users/e4c9453077514f688dbf0c910317381c/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.447723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988208b2-3b8b-451c-9ed3-70acb3d2e75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=863745b4145f48a6b00a4e669753565e, user_id=e4c9453077514f688dbf0c910317381c, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.450634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988208b2-3b8b-451c-9ed3-70acb3d2e75a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.451084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce095661af842779bbb0efe087f5f3f', 'request_id': 'req-757002db-c545-4e7e-803b-7c6edd595fce'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:36.561327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce095661af842779bbb0efe087f5f3f', 'request_id': 'req-757002db-c545-4e7e-803b-7c6edd595fce'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:36.569120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 987/1967] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:36 2022] PUT /identity/v3/projects/863745b4145f48a6b00a4e669753565e/users/e4c9453077514f688dbf0c910317381c/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:36.569998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce095661af842779bbb0efe087f5f3f', 'request_id': 'req-757002db-c545-4e7e-803b-7c6edd595fce'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:36.583884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.584171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.584969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.585091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] The token cache is being invalidate because project fce095661af842779bbb0efe087f5f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:36.585176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:36.585959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-757002db-c545-4e7e-803b-7c6edd595fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:36.588126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 981/1968] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:36 2022] DELETE /identity/v3/projects/fce095661af842779bbb0efe087f5f3f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:36.597104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4aa0a9-5cdd-4e3e-a78b-34d97dfa0a68 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.598208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4aa0a9-5cdd-4e3e-a78b-34d97dfa0a68 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.598208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4aa0a9-5cdd-4e3e-a78b-34d97dfa0a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.632797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-af4aa0a9-5cdd-4e3e-a78b-34d97dfa0a68 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:36.633128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] MFA Rules not processed for user `e4c9453077514f688dbf0c910317381c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:36.634596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.636389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-af4aa0a9-5cdd-4e3e-a78b-34d97dfa0a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.713243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 988/1969] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:36.730373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.731762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.737756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 982/1970] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:36.745161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.745451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.745774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.768143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:36.769697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:36.778848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 983/1971] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:36.783443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.798293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b415-a205-4a5d-8511-d0877466349e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.806359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.806988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.807223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.807585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2ffb6-eafe-4de7-bf4e-53dc739a6c85 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.807709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.808404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:36.808680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.810541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.812087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.812764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.818565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.818565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.822909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.824653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.825535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 984/1972] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:36.830053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:36.840900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 989/1973] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:36.858334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43e52f-1476-4867-bf31-98bb4120fbcd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:36.863218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:36.863829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:36.864089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:36.864357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:36.864974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:36.866919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:36.999278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 985/1974] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:36 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:37.003584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.015872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8e48d-abda-4e62-a475-4263df779644 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.081072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.082127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.082560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.083099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.083887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.086179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.217207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 990/1975] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:37 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:37.219320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.238633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a353e-13e0-4b02-a0da-067301c30d5c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.244926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.245603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.245883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.252568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.252568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.252568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.260639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 986/1976] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:37.265952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.277389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d29e-b035-45d8-94fa-085f125c805a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.287114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadea539-cce2-4063-84eb-7e9b87f5b3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.287114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadea539-cce2-4063-84eb-7e9b87f5b3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.287114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadea539-cce2-4063-84eb-7e9b87f5b3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.287114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadea539-cce2-4063-84eb-7e9b87f5b3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.287501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadea539-cce2-4063-84eb-7e9b87f5b3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.287501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadea539-cce2-4063-84eb-7e9b87f5b3e4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.300917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 991/1977] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:37.305800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.331687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fb011-0d56-4a2e-9742-0b3fff774594 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.336758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.337974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.337974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.338243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] PATH_INFO: `/v3/projects/7e4001643dd44d079973e17bd9ed8978/users/1e57138d32e94f10bbf31717648a2c07/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.357870 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4001643dd44d079973e17bd9ed8978, user_id=1e57138d32e94f10bbf31717648a2c07, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.360710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.495794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 987/1978] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:37 2022] PUT /identity/v3/projects/7e4001643dd44d079973e17bd9ed8978/users/1e57138d32e94f10bbf31717648a2c07/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:37.505776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.506071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.506510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.546357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] MFA Rules not processed for user `1e57138d32e94f10bbf31717648a2c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:37.547663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:37.580125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:37.655413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.655413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:37.667241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 992/1979] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:37.671724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.781010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924644f0-cb76-46e8-8a69-348e7ee8932b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.789375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.791611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.792496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.793159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.794286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.797018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.811476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac6da-e0ec-4acf-b340-bd0afd816599 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.818250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934a08bc45bf468880fce284b5261d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b34e1488736b464ca361d405d2ea58f9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.819266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.819655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.820133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.820942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:37.821321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.823189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.836535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 988/1980] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:37.840629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.859508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6848655-630d-48e5-9c1d-3d9da97d7882 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.865640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934a08bc45bf468880fce284b5261d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b34e1488736b464ca361d405d2ea58f9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.866474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.867342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.867769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.869036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.871577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.898403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 993/1981] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:37 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:37.901803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:37.969785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c1ba-b0a4-409c-a1a4-dc121c2a036a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:37.977026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:37.979710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:37.981130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:37.981130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:37.981130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:37.981374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:37.983341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:37.987948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 989/1982] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:37 2022] POST /identity/v3/projects => generated 354 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:37.995488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.064351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b01dc-0ab4-49c6-b127-69f0ab56ed5d tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.069076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934a08bc45bf468880fce284b5261d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b34e1488736b464ca361d405d2ea58f9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.069681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.069961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.070232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.070899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:38.071199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.073037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.085089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 990/1983] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:14:38.117632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 994/1984] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:37 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:38.127888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.129220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.196606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a2b89-ba0d-419f-9b36-cd485b8458c0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.197396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87847-70c7-419d-ae13-dd6c80b57544 tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.201160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934a08bc45bf468880fce284b5261d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b34e1488736b464ca361d405d2ea58f9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.201801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.202078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.202190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.202376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] PATH_INFO: `/v3/projects/40e4ce847e9e4a4c931dec914ed08733` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.202805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.203087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.203477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.203571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorizing `identity:delete_project(project_id=40e4ce847e9e4a4c931dec914ed08733)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.203995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.205388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.205598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.222701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 991/1985] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:38.226787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.239425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885e41-bf88-41c6-954c-6f446433f253 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.244405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.245072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.245433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.245650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] PATH_INFO: `/v3/projects/a544fe9d7157447b83ed4dee70529b49/users/c113b3c78d9f4eaab236b76fd845ac24/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.261103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a544fe9d7157447b83ed4dee70529b49, user_id=c113b3c78d9f4eaab236b76fd845ac24, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.263836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.419075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 992/1986] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:38 2022] PUT /identity/v3/projects/a544fe9d7157447b83ed4dee70529b49/users/c113b3c78d9f4eaab236b76fd845ac24/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:38.423169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.431698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e4ce847e9e4a4c931dec914ed08733', 'request_id': 'req-97e3a282-d4fb-4501-82f2-c5ec976cc90f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:38.500094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd4d39-751d-4d08-848a-8d96b1e375b8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.505335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.505506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e4ce847e9e4a4c931dec914ed08733', 'request_id': 'req-97e3a282-d4fb-4501-82f2-c5ec976cc90f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:38.506199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.506563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.506960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.507612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.510508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.512341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e4ce847e9e4a4c931dec914ed08733', 'request_id': 'req-97e3a282-d4fb-4501-82f2-c5ec976cc90f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:38.522041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 993/1987] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:38.534845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.535642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] The token cache is being invalidate because project 40e4ce847e9e4a4c931dec914ed08733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:38.535642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:38.538482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97e3a282-d4fb-4501-82f2-c5ec976cc90f tempest-VolumeQuotaClassesTest-1034038489 tempest-VolumeQuotaClassesTest-1034038489-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:38.542002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 995/1988] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:38 2022] DELETE /identity/v3/projects/40e4ce847e9e4a4c931dec914ed08733 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:38.547581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14401a38-b609-411c-a3dc-dd11e97011c3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.552271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.553107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.553378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.553645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] PATH_INFO: `/v3/projects/a544fe9d7157447b83ed4dee70529b49/users/c113b3c78d9f4eaab236b76fd845ac24/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.555629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a544fe9d7157447b83ed4dee70529b49, user_id=c113b3c78d9f4eaab236b76fd845ac24, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.558423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.569034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 994/1989] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:38 2022] PUT /identity/v3/projects/a544fe9d7157447b83ed4dee70529b49/users/c113b3c78d9f4eaab236b76fd845ac24/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:38.582267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.582607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.582934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.619572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] MFA Rules not processed for user `c113b3c78d9f4eaab236b76fd845ac24`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:38.620942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:38.701794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 996/1990] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:38.805825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:38.807656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:38.864033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b32c1a-4bdb-4bb5-b5f3-281a0dcbafae admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:38.867650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a9d3-5079-4929-87e9-e36b9f41ddee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:38.868241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a9d3-5079-4929-87e9-e36b9f41ddee service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:38.869011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a9d3-5079-4929-87e9-e36b9f41ddee service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:38.869276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a9d3-5079-4929-87e9-e36b9f41ddee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:38.879911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4a9d3-5079-4929-87e9-e36b9f41ddee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:38.881366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4a9d3-5079-4929-87e9-e36b9f41ddee service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:38.892742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 995/1991] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:38.957239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:39.129741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:39.549191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:39.550789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:39.625159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d1441-e024-48c6-9cc7-02f7d6432e4d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:39.630894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:39.631652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:39.631937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:39.632216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] PATH_INFO: `/v3/users/727502d004564f1cb0d75a5e02720622` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:39.633378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=727502d004564f1cb0d75a5e02720622)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:39.635929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:39.876106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '727502d004564f1cb0d75a5e02720622'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:39.889137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '727502d004564f1cb0d75a5e02720622'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:39.898424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '727502d004564f1cb0d75a5e02720622'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:39.898424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '727502d004564f1cb0d75a5e02720622'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:39.898646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '727502d004564f1cb0d75a5e02720622'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.003364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:40.008758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 997/1992] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:39 2022] DELETE /identity/v3/users/727502d004564f1cb0d75a5e02720622 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:40.013888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:40.014175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:40.014457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:40.034064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:40.035391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:40.095852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 996/1993] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:40.105707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:40.107260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:40.166079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c8c55-af54-4af9-8c56-6eb84426c1bb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:40.170612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:40.171697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:40.172243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:40.172706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:40.189517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:40.191341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:40.205522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 998/1994] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:40.215125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:40.632955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:40.635268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:40.647146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f066c1-0618-47d3-95b3-fd11fc0638bd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:40.651676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:40.652286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:40.652561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:40.652827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] PATH_INFO: `/v3/users/1e57138d32e94f10bbf31717648a2c07` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:40.653919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e57138d32e94f10bbf31717648a2c07)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:40.655831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:40.747237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:40.749847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:40.766917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1c380-665b-484b-a599-a663643d4d47 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:40.772016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:40.772652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:40.772937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:40.773204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] PATH_INFO: `/v3/projects/857c6684f3c74c14a781cee43615d0cb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:40.774836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857c6684f3c74c14a781cee43615d0cb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:40.776616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:40.843152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e57138d32e94f10bbf31717648a2c07'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.856333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e57138d32e94f10bbf31717648a2c07'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.862997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e57138d32e94f10bbf31717648a2c07'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.864396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e57138d32e94f10bbf31717648a2c07'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.864899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-20a415fc-d5c3-43d5-8f35-555a610097d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e57138d32e94f10bbf31717648a2c07'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.907990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857c6684f3c74c14a781cee43615d0cb', 'request_id': 'req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:40.934885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 997/1995] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:40 2022] DELETE /identity/v3/users/1e57138d32e94f10bbf31717648a2c07 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:40.939194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:40.939482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:40.939757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:40.957788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:40.959026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:41.015759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 998/1996] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:41.023125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857c6684f3c74c14a781cee43615d0cb', 'request_id': 'req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.025729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:41.026517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857c6684f3c74c14a781cee43615d0cb', 'request_id': 'req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.027165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:41.038610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] The token cache is being invalidate because project 857c6684f3c74c14a781cee43615d0cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:41.039007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.041057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.043203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 999/1997] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:40 2022] DELETE /identity/v3/projects/857c6684f3c74c14a781cee43615d0cb => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:41.046861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:41.048681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:41.084290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cec7b8-33dd-494b-aa0a-9a238af9e5ee None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:41.088161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:41.088764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:41.089041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:41.089312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:41.091166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:41.104119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:41.105779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:41.112843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf4cd7-f15a-48d2-9fa4-a5d42b792a34 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:41.115424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 999/1998] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:41.117838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:41.136196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:41.136196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:41.136196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] PATH_INFO: `/v3/users/934a08bc45bf468880fce284b5261d8b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:41.136196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934a08bc45bf468880fce284b5261d8b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:41.136196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:41.203510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:41.436856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934a08bc45bf468880fce284b5261d8b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.464537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934a08bc45bf468880fce284b5261d8b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.472579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934a08bc45bf468880fce284b5261d8b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.473321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934a08bc45bf468880fce284b5261d8b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.476520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934a08bc45bf468880fce284b5261d8b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.512108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:41.562501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1000/1999] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:41 2022] DELETE /identity/v3/users/934a08bc45bf468880fce284b5261d8b => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:41.599985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:41.601467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:41.615918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745874fc-e1c0-4b01-b994-afd71f8cea5c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:41.678741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:41.679371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:41.679651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:41.679922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] PATH_INFO: `/v3/projects/7e4001643dd44d079973e17bd9ed8978` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:41.681088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4001643dd44d079973e17bd9ed8978)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:41.683024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:41.794287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4001643dd44d079973e17bd9ed8978', 'request_id': 'req-c396aa98-202d-42c0-8cfd-364bdbb106c7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.853584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4001643dd44d079973e17bd9ed8978', 'request_id': 'req-c396aa98-202d-42c0-8cfd-364bdbb106c7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.870422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4001643dd44d079973e17bd9ed8978', 'request_id': 'req-c396aa98-202d-42c0-8cfd-364bdbb106c7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.881639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] The token cache is being invalidate because project 7e4001643dd44d079973e17bd9ed8978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:41.881932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.882612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:41.884791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1000/2000] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:41 2022] DELETE /identity/v3/projects/7e4001643dd44d079973e17bd9ed8978 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:41.889066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:41.890663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:41.904914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596b7be-9605-4527-8e26-ee415e611f87 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:41.966832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:41.969463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:41.969463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:41.969463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] PATH_INFO: `/v3/users/c113b3c78d9f4eaab236b76fd845ac24` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:41.969463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c113b3c78d9f4eaab236b76fd845ac24)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:41.971363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:41.999236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.000644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:42.014426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396aa98-202d-42c0-8cfd-364bdbb106c7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.019526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.020782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.021065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.021509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] PATH_INFO: `/v3/projects/b34e1488736b464ca361d405d2ea58f9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.022674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34e1488736b464ca361d405d2ea58f9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.025598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.157268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34e1488736b464ca361d405d2ea58f9', 'request_id': 'req-121a1130-97ce-48cc-8ea1-0013042b0e52'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.157493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c113b3c78d9f4eaab236b76fd845ac24'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.169554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c113b3c78d9f4eaab236b76fd845ac24'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.176223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c113b3c78d9f4eaab236b76fd845ac24'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.176911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c113b3c78d9f4eaab236b76fd845ac24'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.177510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97ab89d4-ea48-41dd-a777-4a4caee32535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c113b3c78d9f4eaab236b76fd845ac24'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.252667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34e1488736b464ca361d405d2ea58f9', 'request_id': 'req-121a1130-97ce-48cc-8ea1-0013042b0e52'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.259758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34e1488736b464ca361d405d2ea58f9', 'request_id': 'req-121a1130-97ce-48cc-8ea1-0013042b0e52'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.271825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] The token cache is being invalidate because project b34e1488736b464ca361d405d2ea58f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:42.272100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.272777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-121a1130-97ce-48cc-8ea1-0013042b0e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.274265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1001/2001] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:41 2022] DELETE /identity/v3/projects/b34e1488736b464ca361d405d2ea58f9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:42.279130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.279417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.279689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.281662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:42.299419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:42.300611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:42.317129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1001/2002] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:41 2022] DELETE /identity/v3/users/c113b3c78d9f4eaab236b76fd845ac24 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:42.362788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1002/2003] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:42.372203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.372579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.372929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.392652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:42.393910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:42.455995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1002/2004] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:42.461349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.471277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be63d13-de33-48b5-8389-d653c087e49e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.476190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.476883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.477055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.477322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.478601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:42.478925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.480502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.493455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1003/2005] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:42.498238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.505911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:42.507973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086bc6e-37c5-4347-98f1-56de8001ae7e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.512826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.513472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.513749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.514013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.516003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.516726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.655851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1003/2006] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:42 2022] POST /identity/v3/projects => generated 407 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:42.660179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.670307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f42d-972c-4ca4-aa92-1d58467202ea admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.725557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.726152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.726430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.726739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.727363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.729547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.741350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.743716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:42.763003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f244d-a1e3-4e00-8a12-d5149ace230f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.767738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.768327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.768603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.768906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] PATH_INFO: `/v3/projects/a544fe9d7157447b83ed4dee70529b49` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.769984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a544fe9d7157447b83ed4dee70529b49)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.771909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.826523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1004/2007] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:42 2022] POST /identity/v3/users => generated 304 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:42.830675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.840034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231dc47-ed9f-4188-b0fa-04b9624317db admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.851933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.851933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.851933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.851933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.852351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.852351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.860781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1005/2008] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:42.864635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.875044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bf8a5-c2ea-4cfc-9481-15b3369fd3d9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.879951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.880530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.880807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.881080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.881704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.883388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.896486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1006/2009] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:42.898402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:42.913661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68962c-b035-4f36-9722-9ad27e0cc65b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:42.915704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a544fe9d7157447b83ed4dee70529b49', 'request_id': 'req-06a01441-c9bd-446e-9c78-21fbba1e575e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:42.973731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ae7f6-ef38-473f-ab70-7d40d05dac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:42.974438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae7f6-ef38-473f-ab70-7d40d05dac5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:42.974864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae7f6-ef38-473f-ab70-7d40d05dac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:42.975212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae7f6-ef38-473f-ab70-7d40d05dac5d admin admin] PATH_INFO: `/v3/projects/73a19dad1a4548e0861a3d0c1c8447de/users/da936221cd2d44849b0a3abae3576588/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:42.989327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ae7f6-ef38-473f-ab70-7d40d05dac5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a19dad1a4548e0861a3d0c1c8447de, user_id=da936221cd2d44849b0a3abae3576588, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:42.991996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ae7f6-ef38-473f-ab70-7d40d05dac5d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:42.996695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a544fe9d7157447b83ed4dee70529b49', 'request_id': 'req-06a01441-c9bd-446e-9c78-21fbba1e575e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:43.002669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a544fe9d7157447b83ed4dee70529b49', 'request_id': 'req-06a01441-c9bd-446e-9c78-21fbba1e575e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:43.014065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] The token cache is being invalidate because project a544fe9d7157447b83ed4dee70529b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:43.014354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:43.015013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-06a01441-c9bd-446e-9c78-21fbba1e575e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:43.016515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1004/2010] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:42 2022] DELETE /identity/v3/projects/a544fe9d7157447b83ed4dee70529b49 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:43.021149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861cf75-3260-4ca6-954e-d9c52ef6677f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.021430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861cf75-3260-4ca6-954e-d9c52ef6677f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.021699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861cf75-3260-4ca6-954e-d9c52ef6677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.040846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5861cf75-3260-4ca6-954e-d9c52ef6677f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:43.042061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5861cf75-3260-4ca6-954e-d9c52ef6677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.049894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1005/2011] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:43.057722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.058006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.058284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.076610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:43.077793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.085690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1006/2012] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:43.089486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1007/2013] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:42 2022] PUT /identity/v3/projects/73a19dad1a4548e0861a3d0c1c8447de/users/da936221cd2d44849b0a3abae3576588/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:43.089951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.100131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f789-59c0-401f-a44b-15516c0c4ec1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.100299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.100593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.104116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.131588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] MFA Rules not processed for user `da936221cd2d44849b0a3abae3576588`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:43.133200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.161506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.162073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.164308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.164308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.164308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:43.164308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.165954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.178197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1007/2014] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:14:43.183760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.194766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0e55-4ca6-4478-a14d-f6f40aed074b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.200269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.201148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.201922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.202083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.202814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.205129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.212069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1008/2015] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:43.216313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.217664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.232492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e1440-3de0-4a8f-a6d6-7d492451f25f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.237367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.237997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.238280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.238552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.239208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.241179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.285379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1008/2016] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:43.289458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.353168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e71b-4bce-4b67-82c6-8143e6375910 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.353701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1009/2017] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/projects => generated 409 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:43.357476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.408846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.409460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.409741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.410008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.410625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.412561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.418813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f2870-8d04-4501-b026-b175432072ef admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.423454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.424057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.424403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.424669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.425316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.427408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.492334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1009/2018] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:43.493145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.504085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c5925-e0ef-4cb6-8bf0-11f77c198a53 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.509893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.510809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.511243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.511523 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.512164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.512276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:43.513845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.520745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1010/2019] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:43.524464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.524878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1010/2020] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:43.528717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.537761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ca35-976d-44ec-b340-36b68dc3c092 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.538542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85615f8-1e72-4b3c-890c-3c52854dffc9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.542632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.543273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.543388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.543725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.543929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.544212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.544307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.544517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.544995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.545156 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.546581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.546805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.557807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1011/2021] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:43.558287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1011/2022] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:43.561962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.566977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.572733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbfbf7-1415-4346-869a-ca832ef96bef admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.572863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd372faa-a354-4ee1-b017-f7701beff9e4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.577332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfedd8-0c7e-44a2-975d-6107610dc94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.577466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.577974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfedd8-0c7e-44a2-975d-6107610dc94e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.578095 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.578250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfedd8-0c7e-44a2-975d-6107610dc94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.578387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.578543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfedd8-0c7e-44a2-975d-6107610dc94e admin admin] PATH_INFO: `/v3/projects/74c85a68e4e9474a9047fbd0833ff8cf/users/cf395efc74db41c3b0986dd0ebcbfe40/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.578644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] PATH_INFO: `/v3/projects/8f393af8e7ea4bef8256c14d14dd4eb4/users/f1fb7d9971b5489087e1f7bae8ab4809/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.592778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfedd8-0c7e-44a2-975d-6107610dc94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c85a68e4e9474a9047fbd0833ff8cf, user_id=cf395efc74db41c3b0986dd0ebcbfe40, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.593954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f393af8e7ea4bef8256c14d14dd4eb4, user_id=f1fb7d9971b5489087e1f7bae8ab4809, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.595756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfedd8-0c7e-44a2-975d-6107610dc94e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.596878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.620815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:43.666795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1012/2023] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:43 2022] PUT /identity/v3/projects/74c85a68e4e9474a9047fbd0833ff8cf/users/cf395efc74db41c3b0986dd0ebcbfe40/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:43.677207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.677425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.677689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.688950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1012/2024] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:43 2022] PUT /identity/v3/projects/8f393af8e7ea4bef8256c14d14dd4eb4/users/f1fb7d9971b5489087e1f7bae8ab4809/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:43.706731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.710164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] MFA Rules not processed for user `cf395efc74db41c3b0986dd0ebcbfe40`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:43.711708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.775689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4a9e-d4e8-432f-aa9e-fd01c7c8801e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.781771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.782369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.782619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1013/2025] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:43.782739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.782933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.783571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.785191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.793619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.796035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1013/2026] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:43.799862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.804394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a4b47-742b-4f3e-a96d-16fd7306d8b7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.810538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df1acb-788c-45f5-8c1b-ef91a44b5fe3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.810745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.811356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.811635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.811912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.812553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.814451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.815440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84fe0-f8ad-49c8-9621-6d96b15eadad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.818706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe0-f8ad-49c8-9621-6d96b15eadad admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.818706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe0-f8ad-49c8-9621-6d96b15eadad admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.818706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe0-f8ad-49c8-9621-6d96b15eadad admin admin] PATH_INFO: `/v3/projects/8f393af8e7ea4bef8256c14d14dd4eb4/users/f1fb7d9971b5489087e1f7bae8ab4809/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.818706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84fe0-f8ad-49c8-9621-6d96b15eadad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f393af8e7ea4bef8256c14d14dd4eb4, user_id=f1fb7d9971b5489087e1f7bae8ab4809, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.821151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84fe0-f8ad-49c8-9621-6d96b15eadad admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:43.831793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1014/2027] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:43 2022] PUT /identity/v3/projects/8f393af8e7ea4bef8256c14d14dd4eb4/users/f1fb7d9971b5489087e1f7bae8ab4809/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:43.842894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.843202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.843477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.874688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] MFA Rules not processed for user `f1fb7d9971b5489087e1f7bae8ab4809`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:43.876062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.898372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1014/2028] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:14:43.899612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.955648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1015/2029] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:14:43.967538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:43.968942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc747-7351-4273-9fb5-2dae3038243d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:43.969185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:43.974453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:43.976773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:43.978204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:43.978972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:43.986189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:43.986189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:43.986189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.003288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:44.038080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176efcd4-0d1b-421f-ab7f-f43fb47aa281 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.088199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.088797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.092456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1015/2030] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:43 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:44.092562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.092562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.094888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:44.103355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b76b24-1696-42ab-9627-9c205e4b4742 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.107936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.108527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.108835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.109112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.109746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.111363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.122160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1016/2031] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:44.126283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:44.136670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffb9c0-90a1-438c-b80a-b36ca69fb21d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.141412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.142076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.142356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.142622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] PATH_INFO: `/v3/projects/8db913fcbd4e46febf6036690b8a8535/users/051a29c5c3c64d6e92da7091b1b0aa99/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.153394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.154930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.157984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db913fcbd4e46febf6036690b8a8535, user_id=051a29c5c3c64d6e92da7091b1b0aa99, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.160647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.165879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1016/2032] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:43 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:44.203250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:44.260495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1017/2033] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:44 2022] PUT /identity/v3/projects/8db913fcbd4e46febf6036690b8a8535/users/051a29c5c3c64d6e92da7091b1b0aa99/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:44.264564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:44.275461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9660e0-7004-4086-824a-3abcb6bab539 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.286018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:44.334646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f5337-cae1-42f4-b770-305061b5afc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.335340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5337-cae1-42f4-b770-305061b5afc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.335618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5337-cae1-42f4-b770-305061b5afc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.335887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5337-cae1-42f4-b770-305061b5afc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.336541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f5337-cae1-42f4-b770-305061b5afc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.338679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f5337-cae1-42f4-b770-305061b5afc1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.349438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1017/2034] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:44.354016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:44.365959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f983fc-738b-473b-94e8-e1efc9468bdd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.372235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.373049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.373435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.373919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] PATH_INFO: `/v3/projects/8db913fcbd4e46febf6036690b8a8535/users/051a29c5c3c64d6e92da7091b1b0aa99/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.376129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db913fcbd4e46febf6036690b8a8535, user_id=051a29c5c3c64d6e92da7091b1b0aa99, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.380415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.410853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1018/2035] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:44 2022] PUT /identity/v3/projects/8db913fcbd4e46febf6036690b8a8535/users/051a29c5c3c64d6e92da7091b1b0aa99/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:44.425773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.426125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.426317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.460211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] MFA Rules not processed for user `051a29c5c3c64d6e92da7091b1b0aa99`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:44.461355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:44.528069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:44.534123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1018/2036] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:44.543084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:44.544771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:44.569311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:44.611203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1007-59b6-43f5-a988-9c82cb1ef3d0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.614925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b13b5-088b-4d7b-b091-d9f9bad4057c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.615529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b13b5-088b-4d7b-b091-d9f9bad4057c service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.615803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b13b5-088b-4d7b-b091-d9f9bad4057c service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.616072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b13b5-088b-4d7b-b091-d9f9bad4057c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.624129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f197e-92dc-4553-bd4b-96a67de1d92d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:44.627749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:44.628341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:44.628616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:44.628887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:44.638450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.639941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.650159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1019/2037] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:14:44.671565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b13b5-088b-4d7b-b091-d9f9bad4057c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:44.673023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b13b5-088b-4d7b-b091-d9f9bad4057c service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:44.681775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1019/2038] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:44 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:45.041256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:45.202059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:45.263084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c9e60-743f-4869-81a1-ee4df3e4c65d service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:45.267588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:45.268202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:45.268478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:45.268746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] PATH_INFO: `/v3/users/cf395efc74db41c3b0986dd0ebcbfe40` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:45.269842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf395efc74db41c3b0986dd0ebcbfe40)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:45.271781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:45.499487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf395efc74db41c3b0986dd0ebcbfe40'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:45.514236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf395efc74db41c3b0986dd0ebcbfe40'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:45.521402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf395efc74db41c3b0986dd0ebcbfe40'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:45.522142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf395efc74db41c3b0986dd0ebcbfe40'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:45.522805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf395efc74db41c3b0986dd0ebcbfe40'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:45.580211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:45.633036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1020/2039] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:45 2022] DELETE /identity/v3/users/cf395efc74db41c3b0986dd0ebcbfe40 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:14:45.637462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:45.637750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:45.638026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:45.657806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:45.659338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:45.723017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1020/2040] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:45.734528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:45.736160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:45.793144 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ec310-7c82-4cae-a344-b1c1368e32a3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:45.796811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:45.797404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:45.797677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:45.797941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:45.806925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:45.808434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:45.818415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1021/2041] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:46.144609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:46.227592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:46.229153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:46.243399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432d8f4-c151-4cfd-81a4-4351a1162b31 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:46.248693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:46.249336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:46.249914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:46.250190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] PATH_INFO: `/v3/projects/74c85a68e4e9474a9047fbd0833ff8cf` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:46.251592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c85a68e4e9474a9047fbd0833ff8cf)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:46.253572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:46.364412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:46.386140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c85a68e4e9474a9047fbd0833ff8cf', 'request_id': 'req-1fec074a-ca31-4b50-b51e-b6020a7985ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.443330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:46.444182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c85a68e4e9474a9047fbd0833ff8cf', 'request_id': 'req-1fec074a-ca31-4b50-b51e-b6020a7985ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.451242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c85a68e4e9474a9047fbd0833ff8cf', 'request_id': 'req-1fec074a-ca31-4b50-b51e-b6020a7985ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.462881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] The token cache is being invalidate because project 74c85a68e4e9474a9047fbd0833ff8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:46.463173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.463847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.476319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1021/2042] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:46 2022] DELETE /identity/v3/projects/74c85a68e4e9474a9047fbd0833ff8cf => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:46.480307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:46.498901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f73c9-f7aa-47a0-8630-d76a5634403c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:46.555423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:46.556695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:46.557133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:46.557410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] PATH_INFO: `/v3/users/051a29c5c3c64d6e92da7091b1b0aa99` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:46.558711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051a29c5c3c64d6e92da7091b1b0aa99)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:46.560923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:46.788439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051a29c5c3c64d6e92da7091b1b0aa99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.799276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051a29c5c3c64d6e92da7091b1b0aa99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.808769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051a29c5c3c64d6e92da7091b1b0aa99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.810145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051a29c5c3c64d6e92da7091b1b0aa99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.810918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-19ff8aec-77d4-4809-b510-2a70496dd851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051a29c5c3c64d6e92da7091b1b0aa99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:46.864283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:46.906635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1022/2043] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:46 2022] DELETE /identity/v3/users/051a29c5c3c64d6e92da7091b1b0aa99 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:46.912249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:47.071762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:47.409313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:47.411169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:47.427209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec074a-ca31-4b50-b51e-b6020a7985ad admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:47.489522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:47.490167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:47.490446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:47.491101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] PATH_INFO: `/v3/projects/8db913fcbd4e46febf6036690b8a8535` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:47.492211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db913fcbd4e46febf6036690b8a8535)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:47.494193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:47.614427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db913fcbd4e46febf6036690b8a8535', 'request_id': 'req-1242b06d-6c09-4736-b271-e4c9ffd25d31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:47.705350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db913fcbd4e46febf6036690b8a8535', 'request_id': 'req-1242b06d-6c09-4736-b271-e4c9ffd25d31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:47.711226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db913fcbd4e46febf6036690b8a8535', 'request_id': 'req-1242b06d-6c09-4736-b271-e4c9ffd25d31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:47.717387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:47.726810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] The token cache is being invalidate because project 8db913fcbd4e46febf6036690b8a8535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:47.727098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:47.727848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1242b06d-6c09-4736-b271-e4c9ffd25d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:47.729259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1022/2044] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:47 2022] DELETE /identity/v3/projects/8db913fcbd4e46febf6036690b8a8535 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:47.734833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:47.735217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:47.735389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:47.754713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:47.755919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:47.797440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:47.818387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1023/2045] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:47.825697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:47.826057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:47.826337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:47.847969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:47.849178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:47.857170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1023/2046] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:47.861636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:47.872359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c28de1-b817-4a65-9de7-75daa9f93bc5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:47.877670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:47.885118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:47.885118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:47.885118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:47.885118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:47.885118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:47.885118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:47.895086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1024/2047] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:14:47.899563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:47.909160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1128e-1d31-4418-9d55-c833fe1acdb6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:47.913929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:47.914721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:47.914868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:47.917231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:47.917938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:47.922267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:48.203226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1024/2048] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:47 2022] POST /identity/v3/projects => generated 385 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:48.214408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:48.224860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642cf6d-7252-4c8c-9f58-e85ede1155e7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:48.279527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:48.280177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.280456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.280730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.281353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:48.283397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:48.375763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1025/2049] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:48 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:48.380026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:48.392707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225823ad-9756-4e91-8824-05fd972d67e7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:48.394994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:48.395599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.395872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.396143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.396807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:48.398406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:48.409160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1025/2050] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:48.413391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:48.423990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d018f-380b-4550-8e38-10625cf6904f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:48.432832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3eb25-833c-43b3-9c7e-5c2a397d708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:48.432832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3eb25-833c-43b3-9c7e-5c2a397d708f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.432832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3eb25-833c-43b3-9c7e-5c2a397d708f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.432832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3eb25-833c-43b3-9c7e-5c2a397d708f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.433215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3eb25-833c-43b3-9c7e-5c2a397d708f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:48.435506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3eb25-833c-43b3-9c7e-5c2a397d708f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:48.446994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1026/2051] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:14:48.451236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:48.453498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:48.467508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4979-a331-4b12-8168-b7dbda47c89c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:48.472661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:48.473316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.473606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.473907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] PATH_INFO: `/v3/projects/82051cbb2e1544a4b66aeb5a6bff74d2/users/98e8c09266904b4e83770d2d3be7e31e/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.489386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82051cbb2e1544a4b66aeb5a6bff74d2, user_id=98e8c09266904b4e83770d2d3be7e31e, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:48.492177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:48.579957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1026/2052] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:48 2022] PUT /identity/v3/projects/82051cbb2e1544a4b66aeb5a6bff74d2/users/98e8c09266904b4e83770d2d3be7e31e/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:48.591206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.591489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.591761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.621403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] MFA Rules not processed for user `98e8c09266904b4e83770d2d3be7e31e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:48.625747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:48.628073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:48.688579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1027/2053] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:48.699825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:48.701582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:48.758377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7298a-bbe3-4243-ba8d-ed506dd8a07d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:48.762028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35befa5b-a0ce-4893-8c21-7d16d1ef3284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:48.762623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35befa5b-a0ce-4893-8c21-7d16d1ef3284 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.762935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35befa5b-a0ce-4893-8c21-7d16d1ef3284 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.763202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35befa5b-a0ce-4893-8c21-7d16d1ef3284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.771482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35befa5b-a0ce-4893-8c21-7d16d1ef3284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:48.772953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35befa5b-a0ce-4893-8c21-7d16d1ef3284 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:48.782339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1027/2054] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:48.804059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:48.842830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:48.870418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d1bd7-75e3-4069-b9fb-0d822db1aeab None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:48.875015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:48.875626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:48.875899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:48.876248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] PATH_INFO: `/v3/users/98e8c09266904b4e83770d2d3be7e31e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:48.877309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e8c09266904b4e83770d2d3be7e31e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:48.879238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:49.096355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e8c09266904b4e83770d2d3be7e31e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:49.109307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e8c09266904b4e83770d2d3be7e31e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:49.121313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e8c09266904b4e83770d2d3be7e31e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:49.124581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e8c09266904b4e83770d2d3be7e31e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:49.124581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e8c09266904b4e83770d2d3be7e31e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:49.208154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1028/2055] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:14:48 2022] DELETE /identity/v3/users/98e8c09266904b4e83770d2d3be7e31e => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:49.213172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:49.213718 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:49.214150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:49.215602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:49.234551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:49.235805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:49.293119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1028/2056] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:49.302863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:49.304243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:49.334575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:49.360425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da92a9-c19c-4f9c-b8b5-8ce3c9e55532 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:49.364481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddcb81-b3fc-4a2f-b5db-6e22a0e22dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:49.365069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddcb81-b3fc-4a2f-b5db-6e22a0e22dca service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:49.365363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddcb81-b3fc-4a2f-b5db-6e22a0e22dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:49.365852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddcb81-b3fc-4a2f-b5db-6e22a0e22dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:49.376229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddcb81-b3fc-4a2f-b5db-6e22a0e22dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:49.377682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddcb81-b3fc-4a2f-b5db-6e22a0e22dca service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:49.388467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1029/2057] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:14:49.799397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:49.802031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:49.825482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3f095-736c-4470-8a01-61cb47587e89 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:49.831413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:49.832132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:49.832512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:49.832698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] PATH_INFO: `/v3/projects/82051cbb2e1544a4b66aeb5a6bff74d2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:49.833967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82051cbb2e1544a4b66aeb5a6bff74d2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:49.835895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:49.981101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:49.997300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82051cbb2e1544a4b66aeb5a6bff74d2', 'request_id': 'req-1c808685-e477-45c8-8219-5c71d754b68d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:50.094595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82051cbb2e1544a4b66aeb5a6bff74d2', 'request_id': 'req-1c808685-e477-45c8-8219-5c71d754b68d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:50.100681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82051cbb2e1544a4b66aeb5a6bff74d2', 'request_id': 'req-1c808685-e477-45c8-8219-5c71d754b68d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:50.112111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] The token cache is being invalidate because project 82051cbb2e1544a4b66aeb5a6bff74d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:14:50.112395 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:50.113016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c808685-e477-45c8-8219-5c71d754b68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:14:50.114549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1029/2058] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:14:49 2022] DELETE /identity/v3/projects/82051cbb2e1544a4b66aeb5a6bff74d2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:50.119937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.120436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.120541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.140730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:50.141940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:50.220774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1030/2059] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:50.228782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.229302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.229302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.250468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:50.251628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:50.260407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1030/2060] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:50.264913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.276175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797aa25-6fa3-48f4-adab-8ec7385a38da None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.280900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.281547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.281830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.282118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.282797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:14:50.283101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:50.284718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:50.297314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1031/2061] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:14:50.301788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.312562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61c101-28d7-463f-b58d-5bd67b2bf247 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.317398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.317998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.318276 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.318550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.319212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:50.321172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:50.422347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1031/2062] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:14:50 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:50.428734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.439313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee09e54-2448-45c7-b25e-1d7c5bcd0205 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.496501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.497119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.497398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.497669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.498297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:50.500736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:50.597199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1032/2063] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:14:50 2022] POST /identity/v3/users => generated 282 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:14:50.601344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.612136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbc1a-e996-48b7-be93-60e308800421 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.616733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.617324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.617800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.617924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.618624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:50.620248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:50.630350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1032/2064] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:50.634776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.644734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172220c-7178-475e-b6c8-082af3595c37 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.649917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db07e2-c718-4043-b4be-c80595b99c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.650491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db07e2-c718-4043-b4be-c80595b99c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.650809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db07e2-c718-4043-b4be-c80595b99c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.651085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db07e2-c718-4043-b4be-c80595b99c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.651724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db07e2-c718-4043-b4be-c80595b99c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:50.653314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db07e2-c718-4043-b4be-c80595b99c94 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:50.663701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1033/2065] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:14:50.667769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.678359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabfadf-2ea0-4fbd-aa47-5aa9b6b217cc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.683031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.683689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.683956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.684233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] PATH_INFO: `/v3/projects/42d658a71ddf4d5b9c22b252c906a97d/users/2b64aa9a28e34cdd9c5eea32cdafdb30/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.699264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d658a71ddf4d5b9c22b252c906a97d, user_id=2b64aa9a28e34cdd9c5eea32cdafdb30, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:50.701924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:50.802212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1033/2066] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:14:50 2022] PUT /identity/v3/projects/42d658a71ddf4d5b9c22b252c906a97d/users/2b64aa9a28e34cdd9c5eea32cdafdb30/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:14:50.812450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b7ec5-093c-44b8-8de8-cc5c46c4808d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.812740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b7ec5-093c-44b8-8de8-cc5c46c4808d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.813042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b7ec5-093c-44b8-8de8-cc5c46c4808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.843890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e95b7ec5-093c-44b8-8de8-cc5c46c4808d None None] MFA Rules not processed for user `2b64aa9a28e34cdd9c5eea32cdafdb30`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:14:50.845360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e95b7ec5-093c-44b8-8de8-cc5c46c4808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:50.869071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:14:50.910239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1034/2067] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:14:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:14:50.921322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:14:50.923062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:14:50.985486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f923a7-ed2d-400c-93f5-d6d95c0a754d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:14:50.989406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e4177-4620-44ac-80fe-9c5567bc8f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:14:50.990015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e4177-4620-44ac-80fe-9c5567bc8f5a service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:14:50.995776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e4177-4620-44ac-80fe-9c5567bc8f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:14:50.995776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e4177-4620-44ac-80fe-9c5567bc8f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:14:50.999893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e4177-4620-44ac-80fe-9c5567bc8f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:14:51.001362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e4177-4620-44ac-80fe-9c5567bc8f5a service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:14:51.010934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1034/2068] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:14:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:00.928105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=16 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:01.006095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:05.681153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ec737-6768-4483-95c0-56d2cb4dd590 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:05.681879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ec737-6768-4483-95c0-56d2cb4dd590 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:05.681879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ec737-6768-4483-95c0-56d2cb4dd590 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:05.683375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1035/2069] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:15:05.712494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:05.713103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:05.713103 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:05.750835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:05.750835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:05.823144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1035/2070] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:15:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:05.834426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ec737-6768-4483-95c0-56d2cb4dd590 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:05.836167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b84ec737-6768-4483-95c0-56d2cb4dd590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:05.853811 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ec737-6768-4483-95c0-56d2cb4dd590 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:05.859002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:05.859692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:05.859972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:05.860333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:05.870694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:05.876166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:05.902218 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1036/2071] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:06.660013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:06.661581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:06.710880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:06.713237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:06.753475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895abe0-0f98-456b-9c2d-94ac76430ef7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:06.758258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:06.758933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:06.759341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:06.759621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] PATH_INFO: `/v3/users/4c86d01c8a794c3c914eaaadce398a6c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:06.760754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c86d01c8a794c3c914eaaadce398a6c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:06.762753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:06.783465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac293-6734-4b95-95de-b03cd5e53573 service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:06.791866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:06.791866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:06.791866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:06.791866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] PATH_INFO: `/v3/users/da936221cd2d44849b0a3abae3576588` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:06.792475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da936221cd2d44849b0a3abae3576588)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:06.794528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:07.028999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c86d01c8a794c3c914eaaadce398a6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.045089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c86d01c8a794c3c914eaaadce398a6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.053057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c86d01c8a794c3c914eaaadce398a6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.058086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c86d01c8a794c3c914eaaadce398a6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.058086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12c3cc69-f4d8-4b9c-babe-997a0e02e26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c86d01c8a794c3c914eaaadce398a6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.066958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da936221cd2d44849b0a3abae3576588'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.079554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da936221cd2d44849b0a3abae3576588'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.086297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da936221cd2d44849b0a3abae3576588'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.087543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da936221cd2d44849b0a3abae3576588'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.088138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da936221cd2d44849b0a3abae3576588'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.202922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1036/2072] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:06 2022] DELETE /identity/v3/users/4c86d01c8a794c3c914eaaadce398a6c => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:07.338358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1037/2073] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:06 2022] DELETE /identity/v3/users/da936221cd2d44849b0a3abae3576588 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:07.343283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:07.343667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:07.343951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:07.363397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:07.364602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:07.374383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:07.425265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1037/2074] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:07.438305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:07.440297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:07.504163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2233d-7b60-43f8-9501-bd8c26dc552c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:07.507731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:07.508912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:07.509440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:07.509704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:07.518754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:07.520200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:07.529861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1038/2075] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:07.582347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:07.584977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:07.614150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144de8-2dbb-4953-abb2-f7ee8d05c840 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:07.624564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:07.625214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:07.625562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:07.625799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] PATH_INFO: `/v3/projects/369e8ab9aa8742c790d227e781105f11` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:07.627287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=369e8ab9aa8742c790d227e781105f11)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:07.629278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:07.762611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369e8ab9aa8742c790d227e781105f11', 'request_id': 'req-f7ed46f0-23e1-4af3-8223-fb01d03795ee'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.847411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369e8ab9aa8742c790d227e781105f11', 'request_id': 'req-f7ed46f0-23e1-4af3-8223-fb01d03795ee'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.856461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369e8ab9aa8742c790d227e781105f11', 'request_id': 'req-f7ed46f0-23e1-4af3-8223-fb01d03795ee'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.869083 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] The token cache is being invalidate because project 369e8ab9aa8742c790d227e781105f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:07.869379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.870077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:07.871615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1038/2076] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:07 2022] DELETE /identity/v3/projects/369e8ab9aa8742c790d227e781105f11 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:07.875788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:07.886736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7446ca-9efd-446f-933f-ae1e6a4ff09f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:07.892183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:07.947926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:07.948618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:07.948892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:07.949160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] PATH_INFO: `/v3/users/18ebe0ccbe664893bfbac3263dfbe11f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:07.950330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ebe0ccbe664893bfbac3263dfbe11f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:07.952405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:08.051588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:08.053024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:08.066671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed46f0-23e1-4af3-8223-fb01d03795ee admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:08.072550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:08.073201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:08.073448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:08.073719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] PATH_INFO: `/v3/projects/73a19dad1a4548e0861a3d0c1c8447de` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:08.074914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a19dad1a4548e0861a3d0c1c8447de)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:08.076804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:08.266402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a19dad1a4548e0861a3d0c1c8447de', 'request_id': 'req-98acc6a9-1f16-4627-b6c9-f871b54a5be8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.276237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ebe0ccbe664893bfbac3263dfbe11f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.292853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ebe0ccbe664893bfbac3263dfbe11f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.300348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ebe0ccbe664893bfbac3263dfbe11f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.301615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ebe0ccbe664893bfbac3263dfbe11f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.303482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ebe0ccbe664893bfbac3263dfbe11f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.379744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a19dad1a4548e0861a3d0c1c8447de', 'request_id': 'req-98acc6a9-1f16-4627-b6c9-f871b54a5be8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.386480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a19dad1a4548e0861a3d0c1c8447de', 'request_id': 'req-98acc6a9-1f16-4627-b6c9-f871b54a5be8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.398384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] The token cache is being invalidate because project 73a19dad1a4548e0861a3d0c1c8447de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:08.398704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.399406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.400986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1039/2077] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:08 2022] DELETE /identity/v3/projects/73a19dad1a4548e0861a3d0c1c8447de => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:08.404919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:08.406246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:08.433927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:08.458307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1039/2078] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:07 2022] DELETE /identity/v3/users/18ebe0ccbe664893bfbac3263dfbe11f => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:08.472147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acc6a9-1f16-4627-b6c9-f871b54a5be8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:08.531437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:08.532076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:08.532365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:08.532637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] PATH_INFO: `/v3/users/f1fb7d9971b5489087e1f7bae8ab4809` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:08.533777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fb7d9971b5489087e1f7bae8ab4809)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:08.535763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:08.712828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:08.714382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:08.730154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b39c2-a3e7-41c2-ad89-d9a0a53371bf admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:08.735868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:08.736494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:08.736910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:08.737211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] PATH_INFO: `/v3/projects/93ad5917f4eb42cebe5633e46fef1558` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:08.738463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ad5917f4eb42cebe5633e46fef1558)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:08.740420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:08.765590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fb7d9971b5489087e1f7bae8ab4809'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.777410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fb7d9971b5489087e1f7bae8ab4809'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.784154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fb7d9971b5489087e1f7bae8ab4809'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.785356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fb7d9971b5489087e1f7bae8ab4809'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.785830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-49af1dbb-cf18-48a8-82fd-2756a1ba62b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fb7d9971b5489087e1f7bae8ab4809'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:08.886335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1040/2079] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:08 2022] DELETE /identity/v3/users/f1fb7d9971b5489087e1f7bae8ab4809 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:08.894525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ad5917f4eb42cebe5633e46fef1558', 'request_id': 'req-b8e53dff-e715-407f-ac1c-ee81c119aad0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.031312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ad5917f4eb42cebe5633e46fef1558', 'request_id': 'req-b8e53dff-e715-407f-ac1c-ee81c119aad0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.042109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ad5917f4eb42cebe5633e46fef1558', 'request_id': 'req-b8e53dff-e715-407f-ac1c-ee81c119aad0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.064652 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] The token cache is being invalidate because project 93ad5917f4eb42cebe5633e46fef1558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:09.065053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.065787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8e53dff-e715-407f-ac1c-ee81c119aad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.067682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1040/2080] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:08 2022] DELETE /identity/v3/projects/93ad5917f4eb42cebe5633e46fef1558 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:09.072460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.072812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.073119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.093408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:09.094637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.151749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:09.176068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1041/2081] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:09.183588 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.184189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.184671 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.210081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:09.212810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.223518 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1041/2082] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:09.228119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:09.239857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e84a2e-daeb-4f9f-8240-1067dd4b07ae None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:09.262617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:09.263274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.263609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.264410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.264525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:09.264835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:09.271372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:09.298895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1042/2083] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:15:09.301754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:09.335901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:09.335901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfc7f6-1322-4a56-b219-2dd502baa3c3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:09.345389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:09.346922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:09.347123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.347684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.348028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.348375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.349068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:09.351084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:09.361863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d402c3-176d-4995-b544-e17e28ead787 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:09.366766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:09.367461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.368300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.368575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] PATH_INFO: `/v3/users/e4c9453077514f688dbf0c910317381c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.369926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c9453077514f688dbf0c910317381c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:09.371889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:09.509610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1042/2084] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/projects => generated 359 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:09.512854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:09.514691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.585680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48705eed-8f4e-44da-bcbe-f74ca8fa68c6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:09.590332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:09.590998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.591277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.591549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] PATH_INFO: `/v3/projects/8f393af8e7ea4bef8256c14d14dd4eb4` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.592652 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f393af8e7ea4bef8256c14d14dd4eb4)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:09.594541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:09.640605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c9453077514f688dbf0c910317381c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.655254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c9453077514f688dbf0c910317381c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.663237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c9453077514f688dbf0c910317381c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.664085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c9453077514f688dbf0c910317381c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.665381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c9453077514f688dbf0c910317381c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.711101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f393af8e7ea4bef8256c14d14dd4eb4', 'request_id': 'req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.787745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1043/2085] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:09 2022] DELETE /identity/v3/users/e4c9453077514f688dbf0c910317381c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:09.791804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:09.791804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.858728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b2c30-df1b-4f4f-a03a-112cdf50d250 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:09.863440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75355b-cca4-412f-9fa8-92a1f89da925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:09.864050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75355b-cca4-412f-9fa8-92a1f89da925 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.864329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75355b-cca4-412f-9fa8-92a1f89da925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.864744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75355b-cca4-412f-9fa8-92a1f89da925 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.865374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75355b-cca4-412f-9fa8-92a1f89da925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:09.867342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75355b-cca4-412f-9fa8-92a1f89da925 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:09.886926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f393af8e7ea4bef8256c14d14dd4eb4', 'request_id': 'req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.893183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f393af8e7ea4bef8256c14d14dd4eb4', 'request_id': 'req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.905200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] The token cache is being invalidate because project 8f393af8e7ea4bef8256c14d14dd4eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:09.905825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.906165 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6997441d-db2b-4fa6-8fe4-a50dc1b3f6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:09.907759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1043/2086] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:09 2022] DELETE /identity/v3/projects/8f393af8e7ea4bef8256c14d14dd4eb4 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:09.912452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1cd33-6812-487e-bc49-2b1578c41b14 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.912736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1cd33-6812-487e-bc49-2b1578c41b14 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.913013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1cd33-6812-487e-bc49-2b1578c41b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.936683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-35b1cd33-6812-487e-bc49-2b1578c41b14 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:09.938002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-35b1cd33-6812-487e-bc49-2b1578c41b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.947242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1044/2087] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:09.948490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93905b9f-7205-428c-9832-67ff05993400 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.948772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93905b9f-7205-428c-9832-67ff05993400 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.949044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93905b9f-7205-428c-9832-67ff05993400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:09.968857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-93905b9f-7205-428c-9832-67ff05993400 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:09.970326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-93905b9f-7205-428c-9832-67ff05993400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.982803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1045/2088] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:09.984114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93905b9f-7205-428c-9832-67ff05993400 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:09.985462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-93905b9f-7205-428c-9832-67ff05993400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:09.995459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1044/2089] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/users => generated 280 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:09.996974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:09.997260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:09.997534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.017879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:10.019138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:10.029517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1045/2090] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:10.030511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.032183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:10.044783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0876a3-ed4b-419f-b053-a5a3588783cb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.049591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.050280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.050572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.050849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.051483 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.053223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.064507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1046/2091] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:10.065187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.066717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93905b9f-7205-428c-9832-67ff05993400 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.071345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.072307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.073201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.073201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.077447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e9cb-63f7-4a51-a462-ddcd76ba4c47 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.083471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.084047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.084706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.084975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.085090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.085407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.086116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:10.087190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.089900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.096488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1046/2092] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:10.097314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.103689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1047/2093] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:15:10.107059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc78bd-5b07-4741-9985-75f3981d477a service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.107478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.113728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.114338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.114703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.114972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.115616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.117286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.118349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6dc83-1166-4b40-ac7b-b02f9e5f0bc2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.126290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.127649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.128346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.128832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.129773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.130678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1047/2094] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:10.132229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.135199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.139384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:10.152696 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6514203-d523-4a51-889c-dbdac232fd65 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.164151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc07c6-35f8-48ce-afe6-8e0a057fc73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.165284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc07c6-35f8-48ce-afe6-8e0a057fc73e admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.166745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc07c6-35f8-48ce-afe6-8e0a057fc73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.167259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc07c6-35f8-48ce-afe6-8e0a057fc73e admin admin] PATH_INFO: `/v3/projects/17c003ef9df04a6396cb9352fe9f25de/users/099281edf32d4b99a2da19d188a532c6/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.187983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc07c6-35f8-48ce-afe6-8e0a057fc73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c003ef9df04a6396cb9352fe9f25de, user_id=099281edf32d4b99a2da19d188a532c6, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.190961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc07c6-35f8-48ce-afe6-8e0a057fc73e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.299682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1048/2095] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:10 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:10.303749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.342029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1048/2096] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:10 2022] PUT /identity/v3/projects/17c003ef9df04a6396cb9352fe9f25de/users/099281edf32d4b99a2da19d188a532c6/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:10.354329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89807-414e-4f2c-824b-e77214000097 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.354559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89807-414e-4f2c-824b-e77214000097 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.354853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89807-414e-4f2c-824b-e77214000097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.381532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a88cf-c624-4a89-be2f-6fc646e408bd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.396245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-77b89807-414e-4f2c-824b-e77214000097 None None] MFA Rules not processed for user `099281edf32d4b99a2da19d188a532c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:10.397199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-77b89807-414e-4f2c-824b-e77214000097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:10.475985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:10.479950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.480530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.480817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.481068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.481705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.483631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.507065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1049/2097] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:10.547526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89807-414e-4f2c-824b-e77214000097 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.549068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-77b89807-414e-4f2c-824b-e77214000097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:10.604730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1049/2098] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:10 2022] POST /identity/v3/users => generated 284 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:10.608981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.617344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89807-414e-4f2c-824b-e77214000097 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.619697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd33eea-2a87-40fa-8496-58fee5e72e0f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.622472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.623457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.624069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.624236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.624954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.625748 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.626148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.626415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.627208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.628859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.634578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.636821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.641056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1050/2099] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:15:10.645080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.652174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1050/2100] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:15:10.659817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9714a-9902-4f16-9d40-fcd7f78c6a03 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.667161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.670095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.670095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.670295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.670398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.670398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.671046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.679575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aa3a5-8cc6-4a6f-967d-e03ac6c4b023 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.686468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1051/2101] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:10.687775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.688972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.689608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.689883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.690150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] PATH_INFO: `/v3/users/099281edf32d4b99a2da19d188a532c6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.691418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099281edf32d4b99a2da19d188a532c6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.693508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.698931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48dc30-6f5b-42ea-af69-ea9e06fb8282 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:10.704605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796678a1-6945-4c7b-a759-9f196bc123f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:10.704842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796678a1-6945-4c7b-a759-9f196bc123f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.705913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796678a1-6945-4c7b-a759-9f196bc123f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.705913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796678a1-6945-4c7b-a759-9f196bc123f6 admin admin] PATH_INFO: `/v3/projects/162c41bb3a8b4d409a4e7d5783873284/users/a694065a08d94060818ada6218d8363b/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.723134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796678a1-6945-4c7b-a759-9f196bc123f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162c41bb3a8b4d409a4e7d5783873284, user_id=a694065a08d94060818ada6218d8363b, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:10.725972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796678a1-6945-4c7b-a759-9f196bc123f6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:10.831266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1052/2102] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:10 2022] PUT /identity/v3/projects/162c41bb3a8b4d409a4e7d5783873284/users/a694065a08d94060818ada6218d8363b/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:10.842006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:10.842356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:10.842735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:10.876712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] MFA Rules not processed for user `a694065a08d94060818ada6218d8363b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:10.877943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:10.901763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099281edf32d4b99a2da19d188a532c6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:10.915537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099281edf32d4b99a2da19d188a532c6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:10.922882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099281edf32d4b99a2da19d188a532c6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:10.923698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099281edf32d4b99a2da19d188a532c6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:10.924375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fd32bc5f-8498-44bc-8a63-afb5e173bf59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099281edf32d4b99a2da19d188a532c6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:10.950878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1053/2103] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:10.955233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:10.956650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.024925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab930fcc-a5fb-4402-b16a-32c8b0900498 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.031555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1051/2104] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:10 2022] DELETE /identity/v3/users/099281edf32d4b99a2da19d188a532c6 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:11.041534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022538-5335-48be-8931-b609720a1fb4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.042825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022538-5335-48be-8931-b609720a1fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.044269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022538-5335-48be-8931-b609720a1fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.117309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-97022538-5335-48be-8931-b609720a1fb4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:11.121340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-97022538-5335-48be-8931-b609720a1fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.124996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.126397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.126847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.127473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.128279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.131073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.140674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1052/2105] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:11.141776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97022538-5335-48be-8931-b609720a1fb4 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.151678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-97022538-5335-48be-8931-b609720a1fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.169555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97022538-5335-48be-8931-b609720a1fb4 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.192171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.193159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.193677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.194136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] PATH_INFO: `/v3/projects/863745b4145f48a6b00a4e669753565e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.196958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=863745b4145f48a6b00a4e669753565e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.199215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.272532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1054/2106] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:10 2022] POST /identity/v3/projects => generated 361 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:11.273540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.277332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.341381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a7a71-f489-4e4c-a230-fac24efa92a3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.345454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.346284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.346752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.347135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.409630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.411116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.424486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1055/2107] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:11.424486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.432646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905b032-5fe7-47d8-a1ed-2a90ad84d21c service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.437164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.437781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.438062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.438409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.438986 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.440934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.460312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863745b4145f48a6b00a4e669753565e', 'request_id': 'req-76d9fae5-26f8-4604-9900-6303869952cd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:11.517724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:11.577702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863745b4145f48a6b00a4e669753565e', 'request_id': 'req-76d9fae5-26f8-4604-9900-6303869952cd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:11.594745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1056/2108] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:11 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:11.595391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863745b4145f48a6b00a4e669753565e', 'request_id': 'req-76d9fae5-26f8-4604-9900-6303869952cd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:11.597304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.619117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909625d-e7c1-458d-a0c7-5272002d5b32 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.623573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] The token cache is being invalidate because project 863745b4145f48a6b00a4e669753565e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:11.623922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:11.624735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-76d9fae5-26f8-4604-9900-6303869952cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:11.626903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1053/2109] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:11 2022] DELETE /identity/v3/projects/863745b4145f48a6b00a4e669753565e => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:11.632326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648e7b7-82d4-4083-8e17-ef0e1120ba75 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.632501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648e7b7-82d4-4083-8e17-ef0e1120ba75 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.632777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648e7b7-82d4-4083-8e17-ef0e1120ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.676132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f648e7b7-82d4-4083-8e17-ef0e1120ba75 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:11.677368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f648e7b7-82d4-4083-8e17-ef0e1120ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.699375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.700142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.700524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.700894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.701642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.703561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.721026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1057/2110] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:11.725673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.728785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.771932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca28cff-bbae-4f80-8024-2f717ea35f56 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.786400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.787166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.787633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.787971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] PATH_INFO: `/v3/projects/7356ee68bd9a41a097f051432173a51f/users/86ec3b3b75084cb0840485a3a01bb3fd/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.821960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7356ee68bd9a41a097f051432173a51f, user_id=86ec3b3b75084cb0840485a3a01bb3fd, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.825120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.834738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1054/2111] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:11.846087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.846392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.846643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.868944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:11.870250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:11.878549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1055/2112] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:11.882990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.892560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2607e2-b0d6-47d9-a931-a84cc3ff1dba None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.897463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.898046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.898319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.898584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.899465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:11.899763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.901355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.913175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1056/2113] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:15:11.917682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:11.928506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e5ada-492f-4c2d-abde-ecfc943e1ae3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:11.942033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:11.942033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:11.942033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:11.942033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:11.942514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:11.943214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:11.951977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:11.963859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1058/2114] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:11 2022] PUT /identity/v3/projects/7356ee68bd9a41a097f051432173a51f/users/86ec3b3b75084cb0840485a3a01bb3fd/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:11.963859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.041315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256475b3-a224-4d71-9397-a800bf5f267d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.045847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.046438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.046744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.047014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.047644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.049231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.060247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1059/2115] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:12.064441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.078062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bead0-fb0a-4d5e-83e6-200413cdc0bb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.083464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade6395-8878-4ffb-b185-7bd96aca573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.084221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade6395-8878-4ffb-b185-7bd96aca573c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.084573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade6395-8878-4ffb-b185-7bd96aca573c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.084914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade6395-8878-4ffb-b185-7bd96aca573c admin admin] PATH_INFO: `/v3/projects/7356ee68bd9a41a097f051432173a51f/users/86ec3b3b75084cb0840485a3a01bb3fd/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.087157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade6395-8878-4ffb-b185-7bd96aca573c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7356ee68bd9a41a097f051432173a51f, user_id=86ec3b3b75084cb0840485a3a01bb3fd, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.090725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade6395-8878-4ffb-b185-7bd96aca573c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.106594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1060/2116] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:12 2022] PUT /identity/v3/projects/7356ee68bd9a41a097f051432173a51f/users/86ec3b3b75084cb0840485a3a01bb3fd/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:12.118120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.118513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.118830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.141809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1057/2117] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:11 2022] POST /identity/v3/projects => generated 399 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:12.144348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.157892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] MFA Rules not processed for user `86ec3b3b75084cb0840485a3a01bb3fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:12.159168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.236859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053c74-ccbc-45d8-88c9-e4dfa66ed784 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.243417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.244243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.244680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.245096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.245724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.249536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.284935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1061/2118] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:12.285890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.288164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.306184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbc9b-f9cb-4327-8522-482eb10e8914 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.310898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.311587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.311861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.312126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] PATH_INFO: `/v3/projects/17c003ef9df04a6396cb9352fe9f25de` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.313274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c003ef9df04a6396cb9352fe9f25de)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.315202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.360433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1058/2119] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:12 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:12.360433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.360433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.421768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7e5b4-0094-4f85-bbd7-1edc9f0e9850 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.425613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.426232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.426510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.426820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.437865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.439426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.449108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1059/2120] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:12.453467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.460182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3948caa-bbda-40db-961c-ba2261df336d service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.464874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.465465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.465746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.466020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.466680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.468267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.480409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1060/2121] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:12.486226 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.496417 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49d0f-8117-4171-8bcf-e8afdad047a7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.503708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c003ef9df04a6396cb9352fe9f25de', 'request_id': 'req-81d10f2b-500c-4c9c-a492-fd3db605d946'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:12.552478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.553089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.553367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.553639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.554279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.556582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.575598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1061/2122] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:12.576658 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.587639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c2b5b-53a5-41b7-b62e-03f240e007ce admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.592399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ee145-bcad-4b94-b998-74dc25f94c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.593049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ee145-bcad-4b94-b998-74dc25f94c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.593327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ee145-bcad-4b94-b998-74dc25f94c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.593595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ee145-bcad-4b94-b998-74dc25f94c97 admin admin] PATH_INFO: `/v3/projects/f95c900f9a8741cab3afc084a090de88/users/55fbb14328594ff2bfdc16ae621161ea/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.603470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c003ef9df04a6396cb9352fe9f25de', 'request_id': 'req-81d10f2b-500c-4c9c-a492-fd3db605d946'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:12.611181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ee145-bcad-4b94-b998-74dc25f94c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95c900f9a8741cab3afc084a090de88, user_id=55fbb14328594ff2bfdc16ae621161ea, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.614385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ee145-bcad-4b94-b998-74dc25f94c97 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.615317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c003ef9df04a6396cb9352fe9f25de', 'request_id': 'req-81d10f2b-500c-4c9c-a492-fd3db605d946'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:12.627913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] The token cache is being invalidate because project 17c003ef9df04a6396cb9352fe9f25de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:12.628479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:12.629274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81d10f2b-500c-4c9c-a492-fd3db605d946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:12.630871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1062/2123] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:12 2022] DELETE /identity/v3/projects/17c003ef9df04a6396cb9352fe9f25de => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:12.636220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8c52e-6a04-49c6-945e-477635ec49c8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.636500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8c52e-6a04-49c6-945e-477635ec49c8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.636785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8c52e-6a04-49c6-945e-477635ec49c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.656222 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fea8c52e-6a04-49c6-945e-477635ec49c8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:12.657417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fea8c52e-6a04-49c6-945e-477635ec49c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.665548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1063/2124] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:12.673451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.673657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.673972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.701494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:12.703096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.714627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1064/2125] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:12.715888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.725779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea312-e5a4-4893-8f24-dbc8d8f2bdb5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.731116 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.731906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.732160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.732444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.733097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:12.733401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.735043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.747825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1065/2126] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:15:12.751841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.768751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1062/2127] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:12 2022] PUT /identity/v3/projects/f95c900f9a8741cab3afc084a090de88/users/55fbb14328594ff2bfdc16ae621161ea/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:12.773196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e519ea8-45cd-44f9-84f9-f780703aaf21 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.774003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:12.782139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.786265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.786823 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.830427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] MFA Rules not processed for user `55fbb14328594ff2bfdc16ae621161ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:12.832063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.860851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.861622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.862323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.862554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.864198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:12.869167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:12.912483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1063/2128] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:12.914380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.916107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.982013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df28a3-9863-40c7-93a1-5829b35098ee None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:12.990302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:12.990889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:12.995847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:12.995847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:12.996108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1066/2129] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:12 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:12.996108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:12.997172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:12.998245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:13.072658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca713-4a7f-49dc-a306-ed1f4792c38a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:13.076859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:13.077694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:13.078075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:13.078611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:13.081787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:13.083505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:13.096216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1064/2130] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:13.100409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:13.122193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c25dd-fc14-4783-9511-09908200895d service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:13.155479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:13.163167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:13.170569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1067/2131] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:13.221418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:13.222043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:13.222321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:13.222837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:13.223650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:13.225688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:13.378962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1065/2132] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:13 2022] POST /identity/v3/users => generated 286 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:13.379711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:13.386813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:13.400308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672cd52-82bb-49fc-a4dc-01cd9160d23d service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:13.405436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:13.406045 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:13.406325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:13.406602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:13.407516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:13.409411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:13.433387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1068/2133] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:13.437933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:13.450089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a7c63-9670-46cc-97b4-7d331905b60b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:13.461092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e58c4-8821-4596-bc90-6f8990137037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:13.462702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e58c4-8821-4596-bc90-6f8990137037 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:13.463056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e58c4-8821-4596-bc90-6f8990137037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:13.463474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e58c4-8821-4596-bc90-6f8990137037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:13.470843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e58c4-8821-4596-bc90-6f8990137037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:13.473171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e58c4-8821-4596-bc90-6f8990137037 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:13.494731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1066/2134] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:13.496300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:13.508954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9bf4-e562-44e5-a3ea-99289b259557 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:13.515876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:13.516561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:13.517462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:13.517744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] PATH_INFO: `/v3/projects/9ac378861f9f4152993b5010687bf904/users/c3ea9fa555214488af112d3ebeb14b43/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:13.535888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac378861f9f4152993b5010687bf904, user_id=c3ea9fa555214488af112d3ebeb14b43, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:13.539070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:13.681710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1069/2135] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:13 2022] PUT /identity/v3/projects/9ac378861f9f4152993b5010687bf904/users/c3ea9fa555214488af112d3ebeb14b43/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:13.692551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:13.692862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:13.693158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:13.706292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:13.726500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] MFA Rules not processed for user `c3ea9fa555214488af112d3ebeb14b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:13.728504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:13.849746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1067/2136] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:14.089893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:14.094688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:14.128879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:14.188693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46ce6e-f674-42c9-a6b8-70a438991aee admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:14.193416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:14.194043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:14.194316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:14.194589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] PATH_INFO: `/v3/users/c5f2e6ec64db4ba3b3db2ca564c26bed` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:14.199357 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f2e6ec64db4ba3b3db2ca564c26bed)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:14.205575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:14.205712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04e36e-164c-4526-a75b-447bc9d4801f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:14.206984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:14.207580 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:14.207857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:14.208151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:14.220603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:14.221954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:14.232123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1068/2137] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:14.434215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f2e6ec64db4ba3b3db2ca564c26bed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:14.447025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f2e6ec64db4ba3b3db2ca564c26bed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:14.453760 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f2e6ec64db4ba3b3db2ca564c26bed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:14.454461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f2e6ec64db4ba3b3db2ca564c26bed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:14.455108 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f688feee-5f6e-4110-8402-dbdb4c6eda48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f2e6ec64db4ba3b3db2ca564c26bed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:14.583741 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1070/2138] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:14 2022] DELETE /identity/v3/users/c5f2e6ec64db4ba3b3db2ca564c26bed => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:14.822073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:14.844001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce2a2a-2731-4386-851e-b67014204d15 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:14.877629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:14.913198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:14.914066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:14.915238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:14.915657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] PATH_INFO: `/v3/projects/78a0df50573341beb4cf1e2480ba6736` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:14.916966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a0df50573341beb4cf1e2480ba6736)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:14.923512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:15.094768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a0df50573341beb4cf1e2480ba6736', 'request_id': 'req-80397219-3406-4c1a-8ed6-5501f931aefc'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:15.117141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:15.236581 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a0df50573341beb4cf1e2480ba6736', 'request_id': 'req-80397219-3406-4c1a-8ed6-5501f931aefc'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:15.242873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a0df50573341beb4cf1e2480ba6736', 'request_id': 'req-80397219-3406-4c1a-8ed6-5501f931aefc'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:15.254806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] The token cache is being invalidate because project 78a0df50573341beb4cf1e2480ba6736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:15.255097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:15.255744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80397219-3406-4c1a-8ed6-5501f931aefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:15.257216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1069/2139] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:14 2022] DELETE /identity/v3/projects/78a0df50573341beb4cf1e2480ba6736 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:15.262909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.263397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.263517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.283866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:15.285070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:15.344504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1071/2140] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:15.353086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.353325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.353595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.372797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:15.374001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:15.381795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1070/2141] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:15.386600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:15.396597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cadc3-4898-4990-8507-9521ecad7964 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:15.401920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:15.402520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.402831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.403102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.403745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:15.404047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:15.405713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:15.417231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1072/2142] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:15:15.422593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:15.431265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08e7a5-ab3f-406b-afa7-a433b32a9863 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:15.435977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:15.436566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.436843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.437113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.437729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:15.439670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:15.613678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1071/2143] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:15 2022] POST /identity/v3/projects => generated 403 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:15.618342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:15.628137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74ffa-e6f3-4275-8ab6-5329b9f84707 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:15.684146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:15.684757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.685037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.685309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.685971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:15.687925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:15.795387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1073/2144] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:15 2022] POST /identity/v3/users => generated 302 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:15.799493 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:15.810151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93277d1f-c7ed-40cd-910f-e9e591ff98d7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:15.814757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:15.815341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.815612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.815878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.816506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:15.818189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:15.828952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1072/2145] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:15.832830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:15.842857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8287e-96bc-4b96-ad67-ce45c9cd1fe2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:15.847758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5b72b-33b2-4e08-9f5a-e84da27e420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:15.848355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5b72b-33b2-4e08-9f5a-e84da27e420f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.848633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5b72b-33b2-4e08-9f5a-e84da27e420f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.848906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5b72b-33b2-4e08-9f5a-e84da27e420f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.849543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5b72b-33b2-4e08-9f5a-e84da27e420f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:15.851174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5b72b-33b2-4e08-9f5a-e84da27e420f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:15.861677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1074/2146] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:15.868594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:15.878854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f39b0e-dabf-45dc-93c1-3dff6e289d1b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:15.884147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:15.884780 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:15.885054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:15.885325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] PATH_INFO: `/v3/projects/edb602e0984a460f8cefcf5a3bd639e4/users/a4bc80214c6645cb8bb3e3b2e93625cf/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:15.900897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb602e0984a460f8cefcf5a3bd639e4, user_id=a4bc80214c6645cb8bb3e3b2e93625cf, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:15.903647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:16.020036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1073/2147] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:15 2022] PUT /identity/v3/projects/edb602e0984a460f8cefcf5a3bd639e4/users/a4bc80214c6645cb8bb3e3b2e93625cf/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:16.034549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161a436-f100-4c45-bfc0-0809e20b834b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:16.034856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161a436-f100-4c45-bfc0-0809e20b834b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:16.035132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161a436-f100-4c45-bfc0-0809e20b834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:16.059701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b161a436-f100-4c45-bfc0-0809e20b834b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:16.061226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b161a436-f100-4c45-bfc0-0809e20b834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:16.138154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1075/2148] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:16.147886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:16.150378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:16.206118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e157b8a-ea93-49c8-a020-2c2e24d65c69 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:16.210265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:16.210951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:16.211881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:16.216034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:16.221761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:16.223247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:16.233314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1074/2149] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:16.460695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:16.625772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:16.772621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:17.168482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:17.871603 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:17.872212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:17.872212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:17.900419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] MFA Rules not processed for user `a4bc80214c6645cb8bb3e3b2e93625cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:17.901711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:17.965717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1076/2150] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:17.970594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:17.980381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae1694-3910-4740-9cf5-45ef120f0e44 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:17.984966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:17.985549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:17.985827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:17.986096 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:17.986746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:17.988647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:18.113309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1075/2151] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:17 2022] POST /identity/v3/projects => generated 403 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:18.118048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:18.128976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88cd8-5d00-48e1-a376-ce48e1e29bf5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:18.178982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:18.196349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:18.196956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:18.197238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:18.202362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:18.202362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:18.202362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:18.369811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1077/2152] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:18 2022] POST /identity/v3/users => generated 308 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:18.373677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:18.379355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:18.385063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de85a5-3811-4246-98a5-4ea512ea56e1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:18.389655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:18.390249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:18.390527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:18.390835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:18.391471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:18.393063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:18.403777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1076/2153] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:18.407940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:18.418338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640fea5-1281-4cce-bd6b-f488ba884976 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:18.423007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:18.423656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:18.423933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:18.424201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] PATH_INFO: `/v3/projects/26dddef140b34dccaa7730e4a7f42541/users/bd8b48f99a16415b81efdca3ecc538f3/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:18.442595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26dddef140b34dccaa7730e4a7f42541, user_id=bd8b48f99a16415b81efdca3ecc538f3, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:18.445285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:18.532864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1078/2154] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:18 2022] PUT /identity/v3/projects/26dddef140b34dccaa7730e4a7f42541/users/bd8b48f99a16415b81efdca3ecc538f3/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:18.536643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:18.547066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8672b-bfca-40d5-a4c7-a7c33e68ece4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:18.603527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847333d-a9bf-4ee5-bd73-988bc06efacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:18.604126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847333d-a9bf-4ee5-bd73-988bc06efacd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:18.604403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847333d-a9bf-4ee5-bd73-988bc06efacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:18.604676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847333d-a9bf-4ee5-bd73-988bc06efacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:18.605312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847333d-a9bf-4ee5-bd73-988bc06efacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:18.607015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847333d-a9bf-4ee5-bd73-988bc06efacd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:18.616681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1077/2155] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:18.621359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:18.631535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27575be4-383e-46ff-a920-771802fc8c40 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:18.636615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:18.637253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:18.637529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:18.637796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] PATH_INFO: `/v3/projects/26dddef140b34dccaa7730e4a7f42541/users/bd8b48f99a16415b81efdca3ecc538f3/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:18.639709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26dddef140b34dccaa7730e4a7f42541, user_id=bd8b48f99a16415b81efdca3ecc538f3, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:18.642308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:18.652326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1079/2156] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:18 2022] PUT /identity/v3/projects/26dddef140b34dccaa7730e4a7f42541/users/bd8b48f99a16415b81efdca3ecc538f3/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:20.126928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:20.126928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:20.126928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:20.165567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] MFA Rules not processed for user `bd8b48f99a16415b81efdca3ecc538f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:20.169305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:20.261694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1078/2157] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:20 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:21.623829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:21.696187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f03dca-dc93-4469-a1f4-1949d1934585 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:21.701168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:21.701789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:21.702066 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:21.702338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] PATH_INFO: `/v3/users/a4bc80214c6645cb8bb3e3b2e93625cf` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:21.703579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4bc80214c6645cb8bb3e3b2e93625cf)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:21.705549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:21.959843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4bc80214c6645cb8bb3e3b2e93625cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:21.982192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4bc80214c6645cb8bb3e3b2e93625cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:21.984574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4bc80214c6645cb8bb3e3b2e93625cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:21.985543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4bc80214c6645cb8bb3e3b2e93625cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:21.986784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4bc80214c6645cb8bb3e3b2e93625cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:22.106452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1080/2158] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:21 2022] DELETE /identity/v3/users/a4bc80214c6645cb8bb3e3b2e93625cf => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:22.422413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:22.433560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059779c-d368-45fb-9002-02af776e7dd8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:22.504315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:22.505240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:22.505869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:22.506180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] PATH_INFO: `/v3/projects/edb602e0984a460f8cefcf5a3bd639e4` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:22.508571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb602e0984a460f8cefcf5a3bd639e4)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:22.510509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:22.539972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:22.683783 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb602e0984a460f8cefcf5a3bd639e4', 'request_id': 'req-71128004-2dbd-4d2d-830e-6a36320473e4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:22.779564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb602e0984a460f8cefcf5a3bd639e4', 'request_id': 'req-71128004-2dbd-4d2d-830e-6a36320473e4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:22.785787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb602e0984a460f8cefcf5a3bd639e4', 'request_id': 'req-71128004-2dbd-4d2d-830e-6a36320473e4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:22.800129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] The token cache is being invalidate because project edb602e0984a460f8cefcf5a3bd639e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:22.800434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:22.801094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:22.802848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1079/2159] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:22 2022] DELETE /identity/v3/projects/edb602e0984a460f8cefcf5a3bd639e4 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:22.807311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:22.809011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:22.821935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc21a0-f676-4b38-a514-1cba50da7abf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:22.838538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:22.881040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:22.881676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:22.881970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:22.882273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] PATH_INFO: `/v3/users/bd8b48f99a16415b81efdca3ecc538f3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:22.883623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8b48f99a16415b81efdca3ecc538f3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:22.885837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:23.107013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8b48f99a16415b81efdca3ecc538f3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:23.119088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8b48f99a16415b81efdca3ecc538f3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:23.128540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8b48f99a16415b81efdca3ecc538f3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:23.129425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8b48f99a16415b81efdca3ecc538f3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:23.130030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8b48f99a16415b81efdca3ecc538f3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:23.274358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1081/2160] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:22 2022] DELETE /identity/v3/users/bd8b48f99a16415b81efdca3ecc538f3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:23.576229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:23.577876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:23.592051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71128004-2dbd-4d2d-830e-6a36320473e4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:23.649571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:23.650105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:23.650378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:23.650644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] PATH_INFO: `/v3/projects/26dddef140b34dccaa7730e4a7f42541` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:23.651764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=26dddef140b34dccaa7730e4a7f42541)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:23.653642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:23.681135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:23.682601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:23.695085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d10ff-6869-4410-afeb-3294b2a328bd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:23.700713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:23.701334 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:23.701610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:23.701878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] PATH_INFO: `/v3/users/a694065a08d94060818ada6218d8363b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:23.703071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a694065a08d94060818ada6218d8363b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:23.704960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:23.834386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26dddef140b34dccaa7730e4a7f42541', 'request_id': 'req-e04a3760-630f-4cea-b81a-80c813f937ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.013866 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26dddef140b34dccaa7730e4a7f42541', 'request_id': 'req-e04a3760-630f-4cea-b81a-80c813f937ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.020104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26dddef140b34dccaa7730e4a7f42541', 'request_id': 'req-e04a3760-630f-4cea-b81a-80c813f937ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.027604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a694065a08d94060818ada6218d8363b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.031656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] The token cache is being invalidate because project 26dddef140b34dccaa7730e4a7f42541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:24.031928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.032565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e04a3760-630f-4cea-b81a-80c813f937ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.034044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1080/2161] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:23 2022] DELETE /identity/v3/projects/26dddef140b34dccaa7730e4a7f42541 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:24.038927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dc704-613b-461b-9ce6-412e8ba6de66 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.039206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dc704-613b-461b-9ce6-412e8ba6de66 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.039473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dc704-613b-461b-9ce6-412e8ba6de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.039969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a694065a08d94060818ada6218d8363b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.047325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a694065a08d94060818ada6218d8363b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.052433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a694065a08d94060818ada6218d8363b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.052433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-365790e4-8f98-4f10-b3cf-1707106b8965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a694065a08d94060818ada6218d8363b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:24.059964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-826dc704-613b-461b-9ce6-412e8ba6de66 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:24.061173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-826dc704-613b-461b-9ce6-412e8ba6de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:24.091683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:24.122180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1081/2162] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:24.129757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.130044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.130320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.149974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1082/2163] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:23 2022] DELETE /identity/v3/users/a694065a08d94060818ada6218d8363b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:24.150435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:24.151679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:24.154180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.154539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.155059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.174173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:24.175393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:24.211556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1082/2164] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:24.215532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.225438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0244c7-aa87-4738-8500-64ede5920e78 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.230601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.231283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.231560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.236203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.236203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:24.236203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.236203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.236753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1083/2165] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:24.246596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1083/2166] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:15:24.247064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.248677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:24.250839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.261383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925262d2-6326-4c0f-8da6-189822e9bb0f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.266179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.266794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.267070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.267391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.267998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.269882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.311945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf0e2-a49f-4c63-85ec-1ce4dc3d1a97 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.316591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.317215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.317497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.317785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.331180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.334261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.351191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1084/2167] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:15:24.419709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1084/2168] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:24.422105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.435575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22d97-5b95-40ab-9434-d8b8c0cda629 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.496248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.496779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.497056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.497362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.498051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.500032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.607417 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1085/2169] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:24.608563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.621729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086349be-476f-4067-8c0d-c305f113acc2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.626553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.627215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.627722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.628000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.628665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.630325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.641013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1085/2170] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:15:24.645543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.656392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1370-a5c8-4198-877b-45602ffd8a37 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.661385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f4230-42ac-4dbc-94d7-53eae7f9e7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.662091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f4230-42ac-4dbc-94d7-53eae7f9e7bf admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.662456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f4230-42ac-4dbc-94d7-53eae7f9e7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.662762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f4230-42ac-4dbc-94d7-53eae7f9e7bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.663403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f4230-42ac-4dbc-94d7-53eae7f9e7bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.666343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f4230-42ac-4dbc-94d7-53eae7f9e7bf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.676373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1086/2171] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:15:24.680919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.695010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ec601-327d-4a3d-a3db-29b9c9bd495b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.696720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.697367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.697647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.697917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] PATH_INFO: `/v3/projects/c23574192ecb4115b7580ddc93d11a3e/users/00026b821a6c434e82c8257df1f8337f/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.712932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23574192ecb4115b7580ddc93d11a3e, user_id=00026b821a6c434e82c8257df1f8337f, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.715619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.831445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1086/2172] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:24 2022] PUT /identity/v3/projects/c23574192ecb4115b7580ddc93d11a3e/users/00026b821a6c434e82c8257df1f8337f/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:24.842742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.843028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.843298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.853998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:24.868510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.870195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:24.877898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] MFA Rules not processed for user `00026b821a6c434e82c8257df1f8337f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:24.879216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:24.936675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4a7b4-b2f8-42d0-a185-bfde0b96e85d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.941286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.941921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.942288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.942704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] PATH_INFO: `/v3/projects/162c41bb3a8b4d409a4e7d5783873284` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.943830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=162c41bb3a8b4d409a4e7d5783873284)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.945753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:24.948374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1087/2173] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:24.953067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:24.964112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7d229-8cdb-411f-a5bf-f122b0487743 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:24.968725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:24.969317 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:24.969593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:24.970145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:24.970876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:24.972795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.061124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162c41bb3a8b4d409a4e7d5783873284', 'request_id': 'req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.084358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1088/2174] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:24 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:25.088808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.149537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47d21c-6f8b-4b57-a7e8-36f7467abdbd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.150846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162c41bb3a8b4d409a4e7d5783873284', 'request_id': 'req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.154322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.154980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.155259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.155533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.156164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.157530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162c41bb3a8b4d409a4e7d5783873284', 'request_id': 'req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.158091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.169536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] The token cache is being invalidate because project 162c41bb3a8b4d409a4e7d5783873284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:25.169826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.170533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.172108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1087/2175] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:24 2022] DELETE /identity/v3/projects/162c41bb3a8b4d409a4e7d5783873284 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:25.176098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.177210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:25.241488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d01eb-ac67-4361-9e8d-b1d7882ca8fc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.246229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.246871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.247153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.247427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] PATH_INFO: `/v3/users/86ec3b3b75084cb0840485a3a01bb3fd` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.248553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ec3b3b75084cb0840485a3a01bb3fd)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.250469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.271035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1089/2176] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:25 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:25.275568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.277213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:25.288972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf234d-0226-4d1a-bb05-d986dbd331fa admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.293981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.294567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.294881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.295158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.295791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.297379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.307107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1090/2177] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:25.311274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.320952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb59fc-997e-461d-9886-ea00e3622f58 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.325501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.326109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.326388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.326673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] PATH_INFO: `/v3/projects/5ba79a50bbc84e68a6882ce26275da1a/users/4632eb4b359448578f65cdaf58236d07/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.341319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba79a50bbc84e68a6882ce26275da1a, user_id=4632eb4b359448578f65cdaf58236d07, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.344072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.420264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1091/2178] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:25 2022] PUT /identity/v3/projects/5ba79a50bbc84e68a6882ce26275da1a/users/4632eb4b359448578f65cdaf58236d07/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:25.426473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.487177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984aa9-9427-4508-a1f3-2b4cf6e1a552 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.491827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.492993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.493274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.493546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.494178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.495841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.505502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1092/2179] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:15:25.509304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ec3b3b75084cb0840485a3a01bb3fd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.510138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.519403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4341920-d1d2-489e-bda7-0af5c606156e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.521471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ec3b3b75084cb0840485a3a01bb3fd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.524466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd06ef-cd30-4e83-8504-8da99211feb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.525098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd06ef-cd30-4e83-8504-8da99211feb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.525373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd06ef-cd30-4e83-8504-8da99211feb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.525636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd06ef-cd30-4e83-8504-8da99211feb2 admin admin] PATH_INFO: `/v3/projects/5ba79a50bbc84e68a6882ce26275da1a/users/4632eb4b359448578f65cdaf58236d07/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.527598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd06ef-cd30-4e83-8504-8da99211feb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba79a50bbc84e68a6882ce26275da1a, user_id=4632eb4b359448578f65cdaf58236d07, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.528946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ec3b3b75084cb0840485a3a01bb3fd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.529670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ec3b3b75084cb0840485a3a01bb3fd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.531344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd06ef-cd30-4e83-8504-8da99211feb2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.531459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ec3b3b75084cb0840485a3a01bb3fd'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:25.540824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1093/2180] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:25 2022] PUT /identity/v3/projects/5ba79a50bbc84e68a6882ce26275da1a/users/4632eb4b359448578f65cdaf58236d07/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:25.551896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.552186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.552460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.580977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] MFA Rules not processed for user `4632eb4b359448578f65cdaf58236d07`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:25.582147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:25.629722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1088/2181] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:25 2022] DELETE /identity/v3/users/86ec3b3b75084cb0840485a3a01bb3fd => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:25.652864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1094/2182] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:25.665764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:25.667376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:25.680155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:25.729141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a11a4-5d0b-4845-9ace-f2fbafb4bace admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:25.732766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:25.733456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:25.733804 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:25.734139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:25.796562 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:25.797838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:25.809377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1089/2183] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:26.084680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:26.137817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452ccae-c2e2-4e8b-86da-7ff17e5f3e90 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:26.141387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:26.141984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.142265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.142537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.144305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:26.158938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:26.160608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:26.168857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:26.209732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:26.215085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7b056-5728-40f5-8329-431a2ccf91d6 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:26.222441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1095/2184] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:26.230357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32acefb-3f43-472b-9cda-d521ee30de0f service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:26.235028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:26.235651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.235930 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.236202 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] PATH_INFO: `/v3/projects/7356ee68bd9a41a097f051432173a51f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.237371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7356ee68bd9a41a097f051432173a51f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:26.239246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:26.261832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:26.416020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7356ee68bd9a41a097f051432173a51f', 'request_id': 'req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:26.515824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7356ee68bd9a41a097f051432173a51f', 'request_id': 'req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:26.522170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7356ee68bd9a41a097f051432173a51f', 'request_id': 'req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:26.533829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] The token cache is being invalidate because project 7356ee68bd9a41a097f051432173a51f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:26.534115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:26.534830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d3806904-bda5-4e79-a42e-f7bbe1b3e92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:26.536369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1090/2185] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:26 2022] DELETE /identity/v3/projects/7356ee68bd9a41a097f051432173a51f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:26.543132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.543438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.543737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.562923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:26.564130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:26.607586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:26.623853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1096/2186] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:26.631733 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.632209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.632351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.652806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:26.654019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:26.661955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1091/2187] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:26.666178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:26.675937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b9da4-0faf-4709-a824-28cbf4a58176 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:26.680491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:26.681206 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.681415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.681678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.682315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:26.682612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:26.684216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:26.697880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1097/2188] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:15:26.706906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:26.719148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b438bae-afca-4610-b0c9-dbc965f68b16 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:26.730366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:26.731503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.736695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.736978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.737683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:26.739860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:26.855704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1092/2189] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:26 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:26.860161 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:26.870848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8ab19-a189-4750-8a1e-a48213b4677e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:26.888275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:26.936091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:26.936701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:26.936983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:26.937255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:26.937885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:26.939831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:27.099950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:27.162487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1098/2190] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:26 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:15:27.166781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:27.176850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51a07f-e7b9-4b48-83a2-9b756dd233eb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:27.183934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:27.184565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:27.201298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:27.201298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:27.201298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:27.201298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:27.201298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1093/2191] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:27.204314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:27.214363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ce819-3183-494a-a878-30f437b433f3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:27.219070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1665ac-61c2-44a4-a956-9cf24674a5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:27.219660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1665ac-61c2-44a4-a956-9cf24674a5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:27.220198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1665ac-61c2-44a4-a956-9cf24674a5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:27.220470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1665ac-61c2-44a4-a956-9cf24674a5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:27.221238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1665ac-61c2-44a4-a956-9cf24674a5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:27.222855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1665ac-61c2-44a4-a956-9cf24674a5e2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:27.233728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1099/2192] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:27.237765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:27.248132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec047-b5a1-4bdc-a20b-5de5a5a6fcfd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:27.252723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:27.253351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:27.253626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:27.253895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] PATH_INFO: `/v3/projects/68fe11308673439cbd7329178004647b/users/4afe373c845b4bb59493f23256aa85a3/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:27.269453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fe11308673439cbd7329178004647b, user_id=4afe373c845b4bb59493f23256aa85a3, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:27.272121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:27.412016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1094/2193] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:27 2022] PUT /identity/v3/projects/68fe11308673439cbd7329178004647b/users/4afe373c845b4bb59493f23256aa85a3/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:27.426137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:27.428011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:27.428011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:27.459189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] MFA Rules not processed for user `4afe373c845b4bb59493f23256aa85a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:27.460339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:27.526428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1100/2194] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:27.530958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:27.590931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f9c89-df69-4077-80ed-39f7bae0f4c7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:27.596214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:27.601944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:27.601944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:27.601944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:27.601944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:27.601944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:27.815577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1095/2195] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:27 2022] POST /identity/v3/projects => generated 381 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:27.833785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:27.845416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24e3e5-5e48-4b1d-b90b-253add70bc74 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:27.853844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:27.901593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:27.902190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:27.902472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:27.902777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:27.903401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:27.905339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.066027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1101/2196] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:27 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:28.070969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.082549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee53e1-c960-4148-b29a-790195ea6e21 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.088206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.088808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.090624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.090624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.090624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.091763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.106531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1096/2197] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:28.110361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.122490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f3c1c-cdbd-4578-883a-966d8450cebd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.127931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.128766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.129418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.130542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] PATH_INFO: `/v3/projects/f7cc09d60cd04313b3ed7a9115468489/users/77257933e2e54c90a099bfef8e061042/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.153868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cc09d60cd04313b3ed7a9115468489, user_id=77257933e2e54c90a099bfef8e061042, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.158288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.259676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1102/2198] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:28 2022] PUT /identity/v3/projects/f7cc09d60cd04313b3ed7a9115468489/users/77257933e2e54c90a099bfef8e061042/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:28.263964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.274976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc1ef-32b2-4010-913d-880ee2265e42 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.331889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fd818-6421-4eb6-99a5-b7c778853faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.332520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd818-6421-4eb6-99a5-b7c778853faa admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.332798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd818-6421-4eb6-99a5-b7c778853faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.333113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd818-6421-4eb6-99a5-b7c778853faa admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.333695 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd818-6421-4eb6-99a5-b7c778853faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.335342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd818-6421-4eb6-99a5-b7c778853faa admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.341274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:28.349785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1097/2199] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:28.351374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.361722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b73e01-402f-4059-ba2b-3ab3fa5e06be admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.366326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.367000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.367824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.368097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] PATH_INFO: `/v3/projects/f7cc09d60cd04313b3ed7a9115468489/users/77257933e2e54c90a099bfef8e061042/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.370018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cc09d60cd04313b3ed7a9115468489, user_id=77257933e2e54c90a099bfef8e061042, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.372754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.386144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1103/2200] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:28 2022] PUT /identity/v3/projects/f7cc09d60cd04313b3ed7a9115468489/users/77257933e2e54c90a099bfef8e061042/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:28.400146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.400461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.401017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.430840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] MFA Rules not processed for user `77257933e2e54c90a099bfef8e061042`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:28.432056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:28.508907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1098/2201] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:28.530054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.531841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:28.538685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.540174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:28.595193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc6a74-e5f9-419b-a6f8-1d8c9c51f185 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.599101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.599816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.600197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.600570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.611070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.612607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.623363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1104/2202] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:28.632759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c35f1-164f-4ad7-84b4-eca15d1e241c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.638102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.639087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.639479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.639859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] PATH_INFO: `/v3/users/c3ea9fa555214488af112d3ebeb14b43` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.641568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3ea9fa555214488af112d3ebeb14b43)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.645437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.812913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:28.823027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc4e6c-2c80-47cd-abdc-c62c13a1f515 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:28.827697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cce04c-25b8-4c3f-b8ee-2bfe24a47cf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:28.828933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce04c-25b8-4c3f-b8ee-2bfe24a47cf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:28.829378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce04c-25b8-4c3f-b8ee-2bfe24a47cf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:28.829813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce04c-25b8-4c3f-b8ee-2bfe24a47cf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:28.894185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cce04c-25b8-4c3f-b8ee-2bfe24a47cf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:28.895654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cce04c-25b8-4c3f-b8ee-2bfe24a47cf9 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:28.905244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1105/2203] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:29.130157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3ea9fa555214488af112d3ebeb14b43'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.145197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3ea9fa555214488af112d3ebeb14b43'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.151354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3ea9fa555214488af112d3ebeb14b43'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.153391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3ea9fa555214488af112d3ebeb14b43'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.153391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3ea9fa555214488af112d3ebeb14b43'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.247659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1099/2204] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:28 2022] DELETE /identity/v3/users/c3ea9fa555214488af112d3ebeb14b43 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:29.259645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:29.260159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:29.260282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:29.269193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:29.281693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:29.282971 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:29.345164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1106/2205] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:29.356479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:29.357832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:29.364710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:29.417795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5541-2698-4db3-a195-6943f37e5d1b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:29.421416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:29.422012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:29.422295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:29.422567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:29.432233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:29.436428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:29.443179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1100/2206] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:29.555770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:29.560806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:29.573065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7a21e-cbf4-41a7-be38-78c391ed3f9e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:29.578449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:29.579258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:29.579635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:29.579979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] PATH_INFO: `/v3/users/4afe373c845b4bb59493f23256aa85a3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:29.582346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4afe373c845b4bb59493f23256aa85a3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:29.584944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:29.918873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:29.920311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:29.933672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250094e-ec06-49e3-9a66-206e1bc7047b service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:29.935913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4afe373c845b4bb59493f23256aa85a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.939532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:29.945579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:29.945579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:29.945579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] PATH_INFO: `/v3/projects/9ac378861f9f4152993b5010687bf904` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:29.945579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac378861f9f4152993b5010687bf904)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:29.945579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:29.951874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4afe373c845b4bb59493f23256aa85a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.958988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4afe373c845b4bb59493f23256aa85a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.959824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4afe373c845b4bb59493f23256aa85a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:29.960469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-65454ef2-cd2c-4905-818f-e1f0b94615a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4afe373c845b4bb59493f23256aa85a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:30.111314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1107/2207] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:29 2022] DELETE /identity/v3/users/4afe373c845b4bb59493f23256aa85a3 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:30.117856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.118140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.118513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.125277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac378861f9f4152993b5010687bf904', 'request_id': 'req-013b9414-894d-4c6b-a25e-ed71480299e1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:30.137719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:30.138995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:30.200409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac378861f9f4152993b5010687bf904', 'request_id': 'req-013b9414-894d-4c6b-a25e-ed71480299e1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:30.201863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1108/2208] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:30.207143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac378861f9f4152993b5010687bf904', 'request_id': 'req-013b9414-894d-4c6b-a25e-ed71480299e1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:30.211795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.213259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:30.218404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] The token cache is being invalidate because project 9ac378861f9f4152993b5010687bf904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:30.218722 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:30.219386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-013b9414-894d-4c6b-a25e-ed71480299e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:30.221134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1101/2209] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:29 2022] DELETE /identity/v3/projects/9ac378861f9f4152993b5010687bf904 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:30.225822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63847d55-85de-456b-be6f-9e7e33879fc5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.230778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63847d55-85de-456b-be6f-9e7e33879fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.230778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63847d55-85de-456b-be6f-9e7e33879fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.233731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:30.246610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-63847d55-85de-456b-be6f-9e7e33879fc5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:30.256544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-63847d55-85de-456b-be6f-9e7e33879fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:30.256544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1102/2210] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:30.265540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.265831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.266120 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.273210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f4224-a0c4-4d6b-88c5-f5377577a94f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.276886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.277492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.277775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.278046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.279905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:30.285426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:30.286590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:30.292649 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.294082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.295157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1103/2211] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:30.299445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.303995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1109/2212] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:15:30.309689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77ad4d-411a-443c-ba57-914ae9e5cd76 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.314872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.315718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.316001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.316368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.317010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:30.317307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.319030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.331269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1104/2213] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:15:30.335661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.345624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b7ef8-13c6-48ed-a4b5-ffe7cd441e4f service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.350480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.351110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.352129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.352407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.353035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.354967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.492882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1110/2214] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:30 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:30.496777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.507821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45464cb-2111-47c9-b0f9-a4a63e7cbeac admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.576896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.577609 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.578110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.578445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.579346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.581497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.591710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:30.705656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1105/2215] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:30 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:30.714812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.721163 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48924-c861-4a86-a297-6812fb61676a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.725989 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.727195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.727477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.727749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.728396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.730010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.740385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1111/2216] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:30.744497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.754850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985a73-21a8-463e-ae88-ca8c9884632d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.759779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.760577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.760878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.761481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.762128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.763743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.775555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1106/2217] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:30.780293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.789919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.790396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:30.803380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621106d-8d26-43b8-93f0-fc942b94141e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.805899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c629b2-7c6d-414c-890c-420bb78cc9ae admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:30.809258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.810328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.810712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.811181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] PATH_INFO: `/v3/projects/5d368aa1f0eb41569a799a30213635af/users/9a6cb9ce9cae4ae3b52a70fffabc170e/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.811541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:30.812237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:30.812582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:30.813116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] PATH_INFO: `/v3/projects/68fe11308673439cbd7329178004647b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:30.814368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68fe11308673439cbd7329178004647b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.816796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.828527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d368aa1f0eb41569a799a30213635af, user_id=9a6cb9ce9cae4ae3b52a70fffabc170e, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:30.831383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:30.975875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1112/2218] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:30 2022] PUT /identity/v3/projects/5d368aa1f0eb41569a799a30213635af/users/9a6cb9ce9cae4ae3b52a70fffabc170e/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:30.977554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:30.979147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.001214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68fe11308673439cbd7329178004647b', 'request_id': 'req-506b7980-a38b-4db6-b19d-5269aaefd063'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.045772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdf0f3-2989-4544-a4d5-9323a035a0d8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:31.100176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:31.100799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:31.101076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:31.101369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] PATH_INFO: `/v3/users/55fbb14328594ff2bfdc16ae621161ea` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:31.102494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fbb14328594ff2bfdc16ae621161ea)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:31.104447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:31.121296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68fe11308673439cbd7329178004647b', 'request_id': 'req-506b7980-a38b-4db6-b19d-5269aaefd063'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.130162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68fe11308673439cbd7329178004647b', 'request_id': 'req-506b7980-a38b-4db6-b19d-5269aaefd063'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.143855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] The token cache is being invalidate because project 68fe11308673439cbd7329178004647b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:31.144139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.144826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-506b7980-a38b-4db6-b19d-5269aaefd063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.146329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1107/2219] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:30 2022] DELETE /identity/v3/projects/68fe11308673439cbd7329178004647b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:31.151011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:31.153413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:31.153413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:31.181903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] MFA Rules not processed for user `9a6cb9ce9cae4ae3b52a70fffabc170e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:31.183072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.247301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1108/2220] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:31.248429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:31.249676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.264036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387f6c-4c6a-4a64-9abd-3cbcdefe81dd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:31.268549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:31.269190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:31.269587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:31.269717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] PATH_INFO: `/v3/users/77257933e2e54c90a099bfef8e061042` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:31.270851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77257933e2e54c90a099bfef8e061042)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:31.272757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:31.387746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fbb14328594ff2bfdc16ae621161ea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.399481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fbb14328594ff2bfdc16ae621161ea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.406260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fbb14328594ff2bfdc16ae621161ea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.406955 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fbb14328594ff2bfdc16ae621161ea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.407558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fbb14328594ff2bfdc16ae621161ea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.564532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1113/2221] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:30 2022] DELETE /identity/v3/users/55fbb14328594ff2bfdc16ae621161ea => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:31.565300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:31.566628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.601275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77257933e2e54c90a099bfef8e061042'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.616036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77257933e2e54c90a099bfef8e061042'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.625076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77257933e2e54c90a099bfef8e061042'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.625843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77257933e2e54c90a099bfef8e061042'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.626447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-454976a3-bed1-484f-a0f9-e0bd72e9d816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77257933e2e54c90a099bfef8e061042'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:31.653589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c45fab-e5bf-4b56-964e-6d098360bfe3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:31.657323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:31.657961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:31.658240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:31.658509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:31.660320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.713958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1109/2222] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:31 2022] DELETE /identity/v3/users/77257933e2e54c90a099bfef8e061042 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:31.717690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:31.718479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:31.718479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:31.730293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:31.731794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:31.742700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1114/2223] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:31.782631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:31.784054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.797668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:31.870000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1110/2224] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:31.882846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:31.884279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:31.944870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4729da-c750-4dad-a683-7ae0eba6f82e service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:32.003713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:32.004709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.005368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.005962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.016290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:32.017745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:32.029223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1115/2225] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:32.170375 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=2,keystone:SELECT=84,keystone:DELETE=14 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:32.463319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:32.463319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:32.477790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfcf6-17f4-47b1-836f-429eb949e664 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:32.508275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:32.509223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.509502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.509774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] PATH_INFO: `/v3/projects/f7cc09d60cd04313b3ed7a9115468489` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.511200 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7cc09d60cd04313b3ed7a9115468489)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:32.513478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:32.668549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7cc09d60cd04313b3ed7a9115468489', 'request_id': 'req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:32.718780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:32.720632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:32.736651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b6fe-05e8-4de1-92aa-58a780e07e33 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:32.746106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7cc09d60cd04313b3ed7a9115468489', 'request_id': 'req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:32.753542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7cc09d60cd04313b3ed7a9115468489', 'request_id': 'req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:32.767245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] The token cache is being invalidate because project f7cc09d60cd04313b3ed7a9115468489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:32.767532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:32.768183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1ef63e1c-ea6c-4a0a-9d03-22ec6349f5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:32.769710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1111/2226] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:32 2022] DELETE /identity/v3/projects/f7cc09d60cd04313b3ed7a9115468489 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:32.775323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251ead-97c6-4e47-bd33-def4065845f3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.775841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251ead-97c6-4e47-bd33-def4065845f3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.776289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251ead-97c6-4e47-bd33-def4065845f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.795707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9e251ead-97c6-4e47-bd33-def4065845f3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:32.796902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9e251ead-97c6-4e47-bd33-def4065845f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:32.799374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:32.800000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.800380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.800755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] PATH_INFO: `/v3/projects/f95c900f9a8741cab3afc084a090de88` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.802011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95c900f9a8741cab3afc084a090de88)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:32.803963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:32.805464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1112/2227] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:32.813551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.813986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.814549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.836302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:32.840220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:32.849242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1113/2228] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:32.853602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:32.887115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb461b1-640c-40f7-b356-7ae712caf4ef None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:32.887115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:32.887115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.887115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.887528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.887528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:15:32.887528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:32.887528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:32.906993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1114/2229] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:15:32.911764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:32.924434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbe293-2c4b-4291-8620-b05565a71909 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:32.937192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95c900f9a8741cab3afc084a090de88', 'request_id': 'req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:32.987840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:32.988467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:32.988744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:32.989012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:32.994803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:32.997039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:33.020746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95c900f9a8741cab3afc084a090de88', 'request_id': 'req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:33.026946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95c900f9a8741cab3afc084a090de88', 'request_id': 'req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:33.038527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] The token cache is being invalidate because project f95c900f9a8741cab3afc084a090de88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:33.038875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:33.039645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:33.041368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1116/2230] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:32 2022] DELETE /identity/v3/projects/f95c900f9a8741cab3afc084a090de88 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:33.101971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1115/2231] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:32 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:33.106431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:33.107943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:33.121703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77481d30-3a44-4c70-b2f5-5a5b70bce5cf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:33.192961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:33.193595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:33.193873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:33.194140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:33.194806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:33.197344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:33.461569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1117/2232] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:33 2022] POST /identity/v3/users => generated 292 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:33.465736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:33.477504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f24b9-c6af-45c0-9ee3-84af19d9f930 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:33.481993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:33.482627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:33.482942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:33.483214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:33.483845 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:33.485422 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:33.495986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1116/2233] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:33.499921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:33.509762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa023d2-2c24-48ea-8a62-65656641faa2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:33.518259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd3888-e7ad-45fe-8c83-2e4f8b89681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:33.518259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd3888-e7ad-45fe-8c83-2e4f8b89681d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:33.518259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd3888-e7ad-45fe-8c83-2e4f8b89681d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:33.518259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd3888-e7ad-45fe-8c83-2e4f8b89681d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:33.518640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd3888-e7ad-45fe-8c83-2e4f8b89681d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:33.518640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd3888-e7ad-45fe-8c83-2e4f8b89681d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:33.528009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1118/2234] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:33.532041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:33.542419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadcbd7-e784-477a-840c-7d70ce32f414 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:33.547410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:33.548053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:33.548330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:33.548606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] PATH_INFO: `/v3/projects/f4beb694e18c4e469ff08cc06fad4738/users/c4cc958dd2324676a403c965faded084/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:33.563555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4beb694e18c4e469ff08cc06fad4738, user_id=c4cc958dd2324676a403c965faded084, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:33.566210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:33.687971 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1117/2235] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:33 2022] PUT /identity/v3/projects/f4beb694e18c4e469ff08cc06fad4738/users/c4cc958dd2324676a403c965faded084/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:33.699542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:33.699824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:33.700097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:33.728376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] MFA Rules not processed for user `c4cc958dd2324676a403c965faded084`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:33.729580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:33.792088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:33.795006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1119/2236] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:33.799439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:33.860483 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f951a-f11f-4524-a8bd-7965a616d1b5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:33.865051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:33.865737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:33.866020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:33.866284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:33.866945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:33.868846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:33.995828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1118/2237] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:15:33 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:33.999928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:34.012112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff89842-f128-4d35-b4ca-17ae3729696c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:34.068671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:34.069283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.069562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.069835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.071573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:34.073057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:34.129312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:34.208455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1120/2238] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:15:33 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:15:34.212988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:34.223473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c96c00-bceb-4cb6-ba45-f6d73e4142ea admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:34.228189 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:34.228781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.229059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.229354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.229989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:34.231628 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:34.243099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1119/2239] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:34.247399 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:34.257265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddb701-3485-415f-b6ca-309e988ee3f7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:34.263167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:34.263845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.264122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.264394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] PATH_INFO: `/v3/projects/7b173e244c6a433490f9eef8e60cec0d/users/319070aade91469fbe1fa96f8638514f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.279688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b173e244c6a433490f9eef8e60cec0d, user_id=319070aade91469fbe1fa96f8638514f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:34.282331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:34.408533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1121/2240] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:34 2022] PUT /identity/v3/projects/7b173e244c6a433490f9eef8e60cec0d/users/319070aade91469fbe1fa96f8638514f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:15:34.412539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:34.422529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7e4d-4c28-454f-8cec-bc4e582381df admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:34.482519 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226dd69b-1b5c-4c81-9001-893dd190f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:34.483207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226dd69b-1b5c-4c81-9001-893dd190f373 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.483350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226dd69b-1b5c-4c81-9001-893dd190f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.483620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226dd69b-1b5c-4c81-9001-893dd190f373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.484255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226dd69b-1b5c-4c81-9001-893dd190f373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:34.485829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226dd69b-1b5c-4c81-9001-893dd190f373 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:34.496284 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1120/2241] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:15:34.500695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:34.512027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1dd54-d8ab-43dc-8753-46b024e6d389 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:34.516799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:34.517504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.517908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.518183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] PATH_INFO: `/v3/projects/7b173e244c6a433490f9eef8e60cec0d/users/319070aade91469fbe1fa96f8638514f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.520205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b173e244c6a433490f9eef8e60cec0d, user_id=319070aade91469fbe1fa96f8638514f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:34.522861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:34.533292 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1122/2242] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:15:34 2022] PUT /identity/v3/projects/7b173e244c6a433490f9eef8e60cec0d/users/319070aade91469fbe1fa96f8638514f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:34.544210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.544500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.544860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.573031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] MFA Rules not processed for user `319070aade91469fbe1fa96f8638514f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:34.574131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:34.608201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:34.640851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1121/2243] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:34.651787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:34.653862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:34.669557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:34.709933 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8eac5-442a-4c56-8bb0-a410c7122ae0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:34.713676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:34.714275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:34.714554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:34.714865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:34.770830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:34.772182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:34.780966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1123/2244] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:34.922634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:43.445822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:43.447440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:43.460107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3eaef-f69b-4c01-b7f6-ed28af19ed43 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:43.463838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb82140-afab-45ba-9ea0-4d8b487a4a2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:43.464435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb82140-afab-45ba-9ea0-4d8b487a4a2e service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:43.467600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb82140-afab-45ba-9ea0-4d8b487a4a2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:43.467600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb82140-afab-45ba-9ea0-4d8b487a4a2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:43.474936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb82140-afab-45ba-9ea0-4d8b487a4a2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:43.476416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb82140-afab-45ba-9ea0-4d8b487a4a2e service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:43.487528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1122/2245] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:44.985443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=58,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:51.902929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:51.972478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91623a9d-789e-4afb-bba6-c19702e6ce23 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:51.977149 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:51.980437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:51.980437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:51.980437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] PATH_INFO: `/v3/users/c4cc958dd2324676a403c965faded084` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:51.980437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4cc958dd2324676a403c965faded084)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:51.983242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:52.209978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4cc958dd2324676a403c965faded084'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:52.222782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4cc958dd2324676a403c965faded084'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:52.229726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4cc958dd2324676a403c965faded084'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:52.230400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4cc958dd2324676a403c965faded084'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:52.231015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4cc958dd2324676a403c965faded084'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:52.362384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1124/2246] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:51 2022] DELETE /identity/v3/users/c4cc958dd2324676a403c965faded084 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:52.367070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:52.367708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:52.367708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:52.389698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:52.392126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:52.459844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1123/2247] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:15:52.469911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:52.471474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:52.529897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf5559-2036-46a3-b201-0c08c6ec2fcb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:52.533491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:52.534094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:52.534737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:52.534737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:52.543981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:52.545496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:52.555551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1125/2248] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:15:52.982071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:52.983705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:52.999313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63cca-6c43-48ea-ae12-21b5aa2d89a4 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:53.007442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:53.008029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:53.008570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:53.008734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] PATH_INFO: `/v3/projects/f4beb694e18c4e469ff08cc06fad4738` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:53.009828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4beb694e18c4e469ff08cc06fad4738)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:53.019077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:53.140959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4beb694e18c4e469ff08cc06fad4738', 'request_id': 'req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.143679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:53.220815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4beb694e18c4e469ff08cc06fad4738', 'request_id': 'req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.228701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4beb694e18c4e469ff08cc06fad4738', 'request_id': 'req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.247913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] The token cache is being invalidate because project f4beb694e18c4e469ff08cc06fad4738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:53.248311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.249107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ea7b6e65-e194-4dee-a62c-edbbaa3e1870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.251044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1124/2249] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:52 2022] DELETE /identity/v3/projects/f4beb694e18c4e469ff08cc06fad4738 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:53.262017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:53.273274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0544d-246b-4e05-8a53-6f6b225f0b11 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:53.297766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:15:53.338206 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:53.338867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:53.339150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:53.339423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] PATH_INFO: `/v3/users/319070aade91469fbe1fa96f8638514f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:53.340538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=319070aade91469fbe1fa96f8638514f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:53.342447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:53.359514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:53.359801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:53.360077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:53.381711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] MFA Rules not processed for user `ffb46379ef204df6bebd58f3bdb78ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:15:53.383635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:53.446171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1125/2250] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:15:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:15:53.702779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319070aade91469fbe1fa96f8638514f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.714548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319070aade91469fbe1fa96f8638514f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.722386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319070aade91469fbe1fa96f8638514f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.723098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319070aade91469fbe1fa96f8638514f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.723719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319070aade91469fbe1fa96f8638514f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:53.836294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1126/2251] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:15:53 2022] DELETE /identity/v3/users/319070aade91469fbe1fa96f8638514f => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:15:54.457079 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:15:54.458564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:15:54.475838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd9f3-511a-4d76-af4e-f9eba637bd40 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:15:54.543526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:15:54.544251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:15:54.544590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:15:54.544924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] PATH_INFO: `/v3/projects/7b173e244c6a433490f9eef8e60cec0d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:15:54.546859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b173e244c6a433490f9eef8e60cec0d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:15:54.549268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:15:54.672925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b173e244c6a433490f9eef8e60cec0d', 'request_id': 'req-6211364f-4d87-463e-b958-a14a6fd66c30'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:54.762156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b173e244c6a433490f9eef8e60cec0d', 'request_id': 'req-6211364f-4d87-463e-b958-a14a6fd66c30'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:54.769877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b173e244c6a433490f9eef8e60cec0d', 'request_id': 'req-6211364f-4d87-463e-b958-a14a6fd66c30'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:54.782412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] The token cache is being invalidate because project 7b173e244c6a433490f9eef8e60cec0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:15:54.782749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:54.783442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6211364f-4d87-463e-b958-a14a6fd66c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:15:54.791914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1126/2252] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:15:54 2022] DELETE /identity/v3/projects/7b173e244c6a433490f9eef8e60cec0d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:04.275778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:04.778408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=10,keystone:SELECT=82,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:34.079042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:34.080541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:34.093363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d35a0-b652-4e1e-a13f-fe9c61b19d4d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:34.148690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:34.149311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:34.149585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:34.149851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] PATH_INFO: `/v3/users/2b64aa9a28e34cdd9c5eea32cdafdb30` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:34.150982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b64aa9a28e34cdd9c5eea32cdafdb30)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:34.152874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:34.448936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b64aa9a28e34cdd9c5eea32cdafdb30'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:34.462071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b64aa9a28e34cdd9c5eea32cdafdb30'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:34.472461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b64aa9a28e34cdd9c5eea32cdafdb30'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:34.473522 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b64aa9a28e34cdd9c5eea32cdafdb30'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:34.474358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b64aa9a28e34cdd9c5eea32cdafdb30'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:34.613647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1127/2253] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:34 2022] DELETE /identity/v3/users/2b64aa9a28e34cdd9c5eea32cdafdb30 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:34.617777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:34.618292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:34.618409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:34.638770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:34.643114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:34.701896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1127/2254] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:34.711553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:34.712957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:34.775547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1980-8ccc-4c45-b2c9-387bc455c056 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:34.779202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:34.779803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:34.780081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:34.780348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:34.789642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:34.792251 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:34.801602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1128/2255] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:35.228862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:35.230323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:35.242614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525221-c540-40dc-87ac-0bf60dcf2e41 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:35.249150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:35.249831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:35.250111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:35.250902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] PATH_INFO: `/v3/projects/42d658a71ddf4d5b9c22b252c906a97d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:35.255022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d658a71ddf4d5b9c22b252c906a97d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:35.255022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:35.347316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:35.348763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:35.376098 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:35.401731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d658a71ddf4d5b9c22b252c906a97d', 'request_id': 'req-d525fcf4-18fd-43a2-aa67-6c55b96885de'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.424708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11e87d-e662-4bd5-bea4-9212f3c4868b service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:35.486050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:35.486679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:35.486968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:35.487509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:35.488972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:35.492918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d658a71ddf4d5b9c22b252c906a97d', 'request_id': 'req-d525fcf4-18fd-43a2-aa67-6c55b96885de'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.499321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d658a71ddf4d5b9c22b252c906a97d', 'request_id': 'req-d525fcf4-18fd-43a2-aa67-6c55b96885de'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.500867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:35.502493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:35.511309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] The token cache is being invalidate because project 42d658a71ddf4d5b9c22b252c906a97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:35.511594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.512131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1129/2256] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:35.512237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d525fcf4-18fd-43a2-aa67-6c55b96885de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.513669 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1128/2257] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:35 2022] DELETE /identity/v3/projects/42d658a71ddf4d5b9c22b252c906a97d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:35.589560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:35.591732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:35.665819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74df26-a9e3-4330-871c-530bbd3911ce service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:35.671372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:35.672014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:35.672939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:35.673218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] PATH_INFO: `/v3/users/00026b821a6c434e82c8257df1f8337f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:35.674400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00026b821a6c434e82c8257df1f8337f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:35.676403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:35.896405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00026b821a6c434e82c8257df1f8337f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.903854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:35.909890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00026b821a6c434e82c8257df1f8337f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.916564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00026b821a6c434e82c8257df1f8337f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.917217 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00026b821a6c434e82c8257df1f8337f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.917825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00026b821a6c434e82c8257df1f8337f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:35.984082 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1130/2258] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:35 2022] DELETE /identity/v3/users/00026b821a6c434e82c8257df1f8337f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:35.989160 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:35.989682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:35.989799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:36.008480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:36.009685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:36.069499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1129/2259] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:36.079141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:36.080624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:36.136693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871d79-5252-40a1-955e-a6487eb9a292 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:36.140275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:36.141385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:36.141667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:36.141951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:36.151642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:36.153060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:36.163153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1131/2260] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:36.590936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:36.592411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:36.608271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdebd3-fb3b-4d3e-8032-86029a627bbd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:36.613159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:36.619139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:36.619139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:36.619139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] PATH_INFO: `/v3/projects/c23574192ecb4115b7580ddc93d11a3e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:36.619139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23574192ecb4115b7580ddc93d11a3e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:36.619139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:36.638649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:36.767034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23574192ecb4115b7580ddc93d11a3e', 'request_id': 'req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:36.823762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23574192ecb4115b7580ddc93d11a3e', 'request_id': 'req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:36.829612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23574192ecb4115b7580ddc93d11a3e', 'request_id': 'req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:36.840600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] The token cache is being invalidate because project c23574192ecb4115b7580ddc93d11a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:36.840884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:36.841521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:36.843076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1130/2261] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:36 2022] DELETE /identity/v3/projects/c23574192ecb4115b7580ddc93d11a3e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:36.848000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:36.857392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559a8b-f7a5-4324-927c-7a94a5addceb service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:36.916652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:36.917632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:36.917632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:36.917842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] PATH_INFO: `/v3/users/4632eb4b359448578f65cdaf58236d07` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:36.918973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4632eb4b359448578f65cdaf58236d07)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:36.920927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:37.174936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4632eb4b359448578f65cdaf58236d07'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:37.186380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4632eb4b359448578f65cdaf58236d07'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:37.193099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4632eb4b359448578f65cdaf58236d07'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:37.193759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4632eb4b359448578f65cdaf58236d07'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:37.194353 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ef55d4ef-0ede-414e-a3bd-b4c0f4d98e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4632eb4b359448578f65cdaf58236d07'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:37.198065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:37.286557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1132/2262] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:36 2022] DELETE /identity/v3/users/4632eb4b359448578f65cdaf58236d07 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:37.690409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:37.692013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:37.705186 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365c1f-5651-4a6d-bdcb-34c76c63ca4e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:37.762274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:37.762919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:37.763191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:37.763459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] PATH_INFO: `/v3/projects/5ba79a50bbc84e68a6882ce26275da1a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:37.764556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba79a50bbc84e68a6882ce26275da1a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:37.766542 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:37.878257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba79a50bbc84e68a6882ce26275da1a', 'request_id': 'req-9e63bcb8-248a-44a1-ad38-f313dda86dc2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:38.029324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba79a50bbc84e68a6882ce26275da1a', 'request_id': 'req-9e63bcb8-248a-44a1-ad38-f313dda86dc2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:38.036299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba79a50bbc84e68a6882ce26275da1a', 'request_id': 'req-9e63bcb8-248a-44a1-ad38-f313dda86dc2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:38.047515 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] The token cache is being invalidate because project 5ba79a50bbc84e68a6882ce26275da1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:38.047886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:38.049050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e63bcb8-248a-44a1-ad38-f313dda86dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:38.050558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1131/2263] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:37 2022] DELETE /identity/v3/projects/5ba79a50bbc84e68a6882ce26275da1a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:38.055462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.055947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.056042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.074906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:38.076121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:38.134386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1133/2264] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:38.143440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.143723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.144024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.163618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:38.164813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:38.172625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1132/2265] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:16:38.177465 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.187041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd485606-604b-4a7b-a0e6-07da4cb0522a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.191572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.192159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.192432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.192700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.193360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:16:38.193658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.195274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.211957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1134/2266] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:16:38.215111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.224983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fba0b1-6534-4330-b1ed-fdb2fecb0042 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.229601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.230203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.230480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.230859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.231469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.233458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.328928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1133/2267] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:38.333259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.343301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a768e5-4bf4-426a-9fe9-44af2b527549 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.398873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.399504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.399782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.400048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.400688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.402562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.487771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1135/2268] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:38.491836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.502142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c88596-6536-4875-ac51-bb7883f2a1e4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.506807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.507389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.507662 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.507969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.508608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.510215 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.512272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:38.520872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1134/2269] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:38.524737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.535332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec95ce8-56d4-4154-92ed-1c4d4646c79d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.540411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c23aeb-fd81-4643-9dea-30ed2885955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.541018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c23aeb-fd81-4643-9dea-30ed2885955f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.542398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c23aeb-fd81-4643-9dea-30ed2885955f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.542398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c23aeb-fd81-4643-9dea-30ed2885955f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.542398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c23aeb-fd81-4643-9dea-30ed2885955f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.543887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c23aeb-fd81-4643-9dea-30ed2885955f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.554574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1136/2270] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:38.558810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.568663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf762f-eee7-4eb3-be48-3b0cd995e8b5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.573876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.582781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.582781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.582781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] PATH_INFO: `/v3/projects/c720e224e12c4971bbae8ff5aa9e0434/users/ce40eabb755f42ad8fdfa5f846bc94f2/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.589861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c720e224e12c4971bbae8ff5aa9e0434, user_id=ce40eabb755f42ad8fdfa5f846bc94f2, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.592798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.692219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1135/2271] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:38 2022] PUT /identity/v3/projects/c720e224e12c4971bbae8ff5aa9e0434/users/ce40eabb755f42ad8fdfa5f846bc94f2/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:38.702591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.702889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.703162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.731288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] MFA Rules not processed for user `ce40eabb755f42ad8fdfa5f846bc94f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:38.732574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:38.779428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:38.795739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1137/2272] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:38.800279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.861271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39192ce5-42fa-482e-bd43-759657b46f92 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:38.866023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:38.866619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:38.866927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:38.867199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:38.867824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:38.869718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:38.973896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1136/2273] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:38.977887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:38.993310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bad727-2906-4750-9e3f-7fb9aad7fe43 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:39.051086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:39.051624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.051901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.052171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.052778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:39.054695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:39.201932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1138/2274] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:38 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:39.205965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:39.216841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072fd45-2537-4994-a327-45286d079533 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:39.221635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:39.222229 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.222500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.222838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.223517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:39.225110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:39.236689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1137/2275] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:39.240879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:39.250644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79383830-2d9b-4876-b6fe-550ec99e5e8a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:39.255999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:39.256637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.261581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.261581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] PATH_INFO: `/v3/projects/e0ad3c69a87047bf90d890bad6c0ff5a/users/8b664f7184994c3dae838c009a4c03ec/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.271767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ad3c69a87047bf90d890bad6c0ff5a, user_id=8b664f7184994c3dae838c009a4c03ec, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:39.274788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:39.345803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1139/2276] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:39 2022] PUT /identity/v3/projects/e0ad3c69a87047bf90d890bad6c0ff5a/users/8b664f7184994c3dae838c009a4c03ec/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:39.349736 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:39.359734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2dcc3-89a0-4c61-bc76-840b4b2624fd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:39.416917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96a525-0302-4984-a5c2-747dbb59f4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:39.417830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96a525-0302-4984-a5c2-747dbb59f4da admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.418107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96a525-0302-4984-a5c2-747dbb59f4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.418368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96a525-0302-4984-a5c2-747dbb59f4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.419302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96a525-0302-4984-a5c2-747dbb59f4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:39.422236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96a525-0302-4984-a5c2-747dbb59f4da admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:39.432473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1138/2277] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:39.436729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:39.447662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cea0b-21ea-416d-a1b4-271681c14a9f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:39.452386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:39.453028 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.453309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.453584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] PATH_INFO: `/v3/projects/e0ad3c69a87047bf90d890bad6c0ff5a/users/8b664f7184994c3dae838c009a4c03ec/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.455518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ad3c69a87047bf90d890bad6c0ff5a, user_id=8b664f7184994c3dae838c009a4c03ec, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:39.458164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:39.468315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1140/2278] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:39 2022] PUT /identity/v3/projects/e0ad3c69a87047bf90d890bad6c0ff5a/users/8b664f7184994c3dae838c009a4c03ec/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:39.479046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.479330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.479599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.507813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] MFA Rules not processed for user `8b664f7184994c3dae838c009a4c03ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:39.509009 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:39.543545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:39.578677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1139/2279] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:39.591178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:39.592881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:39.652937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4f51-6d22-4838-9c2e-12cac8a0ec1b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:39.656642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15a436-5b77-4e42-9bc7-0d93837952e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:39.657743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a436-5b77-4e42-9bc7-0d93837952e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:39.658020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a436-5b77-4e42-9bc7-0d93837952e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:39.658288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a436-5b77-4e42-9bc7-0d93837952e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:39.667090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15a436-5b77-4e42-9bc7-0d93837952e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:39.668512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15a436-5b77-4e42-9bc7-0d93837952e5 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:39.678442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1141/2280] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:40.492731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:40.553294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de38a8-1b61-483e-9d86-c1e2d571b262 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:40.557749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:40.558359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:40.558640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:40.558943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] PATH_INFO: `/v3/users/ce40eabb755f42ad8fdfa5f846bc94f2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:40.560052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce40eabb755f42ad8fdfa5f846bc94f2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:40.561957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:40.773720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce40eabb755f42ad8fdfa5f846bc94f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:40.785462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce40eabb755f42ad8fdfa5f846bc94f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:40.792231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce40eabb755f42ad8fdfa5f846bc94f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:40.792911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce40eabb755f42ad8fdfa5f846bc94f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:40.793512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce40eabb755f42ad8fdfa5f846bc94f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:40.875457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1140/2281] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:40 2022] DELETE /identity/v3/users/ce40eabb755f42ad8fdfa5f846bc94f2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:16:40.880473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:40.880958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:40.881077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:40.899519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:40.900744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:40.951275 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:40.968303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1142/2282] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:40.978298 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:40.979789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:41.043330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:41.047040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32c039-99d6-475b-aa07-ea840240eeff admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:41.051261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:41.051857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:41.060666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:41.060666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:41.061441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:41.062910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:41.072147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1141/2283] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:41.540664 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:41.542214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:41.555019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b058ba-b17e-4d34-ad67-9c0fa967104c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:41.559980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:41.560615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:41.565546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:41.565546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] PATH_INFO: `/v3/projects/c720e224e12c4971bbae8ff5aa9e0434` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:41.565546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c720e224e12c4971bbae8ff5aa9e0434)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:41.565546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:41.676121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c720e224e12c4971bbae8ff5aa9e0434', 'request_id': 'req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:41.737987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c720e224e12c4971bbae8ff5aa9e0434', 'request_id': 'req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:41.744240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c720e224e12c4971bbae8ff5aa9e0434', 'request_id': 'req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:41.756012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] The token cache is being invalidate because project c720e224e12c4971bbae8ff5aa9e0434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:41.756301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:41.756996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:41.758936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1143/2284] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:41 2022] DELETE /identity/v3/projects/c720e224e12c4971bbae8ff5aa9e0434 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:41.763181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:41.773042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b386-baf1-409c-b0c5-253c11a5032e service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:41.830251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:41.830834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:41.831116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:41.831392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] PATH_INFO: `/v3/users/8b664f7184994c3dae838c009a4c03ec` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:41.832501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b664f7184994c3dae838c009a4c03ec)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:41.834430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:42.043343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b664f7184994c3dae838c009a4c03ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.055548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b664f7184994c3dae838c009a4c03ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.062290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b664f7184994c3dae838c009a4c03ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.063000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b664f7184994c3dae838c009a4c03ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.063608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4134f3fe-9a38-403d-9191-70a1e452fb7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b664f7184994c3dae838c009a4c03ec'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.149904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1142/2285] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:41 2022] DELETE /identity/v3/users/8b664f7184994c3dae838c009a4c03ec => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:16:42.609005 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:42.610481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:42.626160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc738-2c77-4ce7-8804-3d2b73c0e42c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:42.687231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:42.687866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:42.688144 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:42.688917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] PATH_INFO: `/v3/projects/e0ad3c69a87047bf90d890bad6c0ff5a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:42.690200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ad3c69a87047bf90d890bad6c0ff5a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:42.693025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:42.786289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ad3c69a87047bf90d890bad6c0ff5a', 'request_id': 'req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.847188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ad3c69a87047bf90d890bad6c0ff5a', 'request_id': 'req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.854437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ad3c69a87047bf90d890bad6c0ff5a', 'request_id': 'req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.863055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:42.868147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] The token cache is being invalidate because project e0ad3c69a87047bf90d890bad6c0ff5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:42.868555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.869610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-42ae2a9e-725b-4ccf-9bbe-6f7251e65d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:42.876703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1144/2286] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:42 2022] DELETE /identity/v3/projects/e0ad3c69a87047bf90d890bad6c0ff5a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:42.878159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:42.878689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:42.878803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:42.898893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:42.900149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:42.960675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1143/2287] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:42.968703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:42.969249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:42.969639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:42.989430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:42.990624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:42.998979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1145/2288] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:43.004082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.013740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003fb33-a88b-425f-b034-423691b1d97f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.020365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.020798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.021071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.021336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.022606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:16:43.022606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.023964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.031035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:43.036182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1144/2289] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:16:43.040617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.050607 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aedbe3-be28-42dd-8ced-3d9a6182b713 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.055332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.055918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.056190 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.056453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.057255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.059165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.146365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1146/2290] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:43 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:43.150482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.161239 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0213-1d93-40b1-8a37-d1f2c9424a71 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.215488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.215997 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.216274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.216539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.217188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.219560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.311293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1145/2291] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:43 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:16:43.315881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.326054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89682e55-40dc-4070-8254-652d222ba50c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.331263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.331861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.332140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.332414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.333051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.334630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.344630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1147/2292] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:43.348668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.358505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c08e56-bfb6-4c29-817c-632d1e4e520c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.363905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195e26e-47b1-46c0-97ae-2d2d64ba6a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.364523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195e26e-47b1-46c0-97ae-2d2d64ba6a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.364797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195e26e-47b1-46c0-97ae-2d2d64ba6a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.365069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195e26e-47b1-46c0-97ae-2d2d64ba6a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.365709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195e26e-47b1-46c0-97ae-2d2d64ba6a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.367349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195e26e-47b1-46c0-97ae-2d2d64ba6a91 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.378428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1146/2293] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:43.382584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.393261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6ef03-c0c7-4ae1-a2b3-d721762402e6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.398017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.398742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.399023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.399297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] PATH_INFO: `/v3/projects/4b1e369f3fb74aa5b1e7fd2d71468136/users/5b6d5a5a65d9463cb4a8e83f5573aa77/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.415171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e369f3fb74aa5b1e7fd2d71468136, user_id=5b6d5a5a65d9463cb4a8e83f5573aa77, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.417906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.504493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1148/2294] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:43 2022] PUT /identity/v3/projects/4b1e369f3fb74aa5b1e7fd2d71468136/users/5b6d5a5a65d9463cb4a8e83f5573aa77/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:43.515576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.516312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.516592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.553077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] MFA Rules not processed for user `5b6d5a5a65d9463cb4a8e83f5573aa77`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:43.554525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:43.625438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1147/2295] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:43.630892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.692100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bdbb4-a878-4c20-a7a8-972206f6f2ae admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.696775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.697370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.697647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.697919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.698579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.700506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.768018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1149/2296] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:43 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:43.771946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.782541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354f89-59b1-45f7-8ee5-30ee6814127b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.837954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.838552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.838863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.839135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.839768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.841673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.853133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:43.939017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1148/2297] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:43 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:43.947752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.959743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228e5dc-82b2-4ec5-82a4-f6baef943f9b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:43.964589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:43.965238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:43.965516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:43.965785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:43.966418 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:43.968059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:43.980879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1150/2298] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:43.985303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:43.995441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939801a-b7ad-4ace-bfbe-664fb6f87a45 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:44.000087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:44.001525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:44.002002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:44.002328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] PATH_INFO: `/v3/projects/9d217fc9950b4005a0c9b36efdd4685b/users/a6abe88bd7634559848d36280713e14c/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:44.018296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d217fc9950b4005a0c9b36efdd4685b, user_id=a6abe88bd7634559848d36280713e14c, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:44.020893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:44.087025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1149/2299] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:43 2022] PUT /identity/v3/projects/9d217fc9950b4005a0c9b36efdd4685b/users/a6abe88bd7634559848d36280713e14c/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:44.091166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:44.101584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992de2b-abe4-4561-bc06-d7f2c7babd7b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:44.168215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234516e5-ae5d-45f1-8607-02aa79435da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:44.168215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234516e5-ae5d-45f1-8607-02aa79435da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:44.168215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234516e5-ae5d-45f1-8607-02aa79435da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:44.168215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234516e5-ae5d-45f1-8607-02aa79435da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:44.168688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234516e5-ae5d-45f1-8607-02aa79435da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:44.168688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234516e5-ae5d-45f1-8607-02aa79435da6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:44.172793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1151/2300] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:44.177343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:44.188639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565c5fd-fc15-4636-8964-cf46528380e4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:44.193502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:44.194182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:44.194460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:44.194771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] PATH_INFO: `/v3/projects/9d217fc9950b4005a0c9b36efdd4685b/users/a6abe88bd7634559848d36280713e14c/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:44.196818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d217fc9950b4005a0c9b36efdd4685b, user_id=a6abe88bd7634559848d36280713e14c, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:44.199624 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:44.210704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1150/2301] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:44 2022] PUT /identity/v3/projects/9d217fc9950b4005a0c9b36efdd4685b/users/a6abe88bd7634559848d36280713e14c/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:44.222201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:44.222554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:44.222958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:44.236738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:44.250358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] MFA Rules not processed for user `a6abe88bd7634559848d36280713e14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:44.252113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:44.324905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1152/2302] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:44.336274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:44.338027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:44.397195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f59d7-28db-45ef-a195-04b2aa983cc2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:44.400968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e7ce5-6551-414d-845f-9bdadf57710f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:44.401566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7ce5-6551-414d-845f-9bdadf57710f service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:44.401847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7ce5-6551-414d-845f-9bdadf57710f service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:44.402104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e7ce5-6551-414d-845f-9bdadf57710f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:44.460021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7ce5-6551-414d-845f-9bdadf57710f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:44.461602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e7ce5-6551-414d-845f-9bdadf57710f service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:44.470836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1151/2303] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:45.193804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:45.195373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:45.261443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627d555-26dd-4348-b6cd-d0f7df964e28 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:45.265947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:45.267216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:45.267472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:45.267733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] PATH_INFO: `/v3/users/9a6cb9ce9cae4ae3b52a70fffabc170e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:45.268847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6cb9ce9cae4ae3b52a70fffabc170e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:45.270801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:45.484047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6cb9ce9cae4ae3b52a70fffabc170e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:45.496343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6cb9ce9cae4ae3b52a70fffabc170e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:45.503894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6cb9ce9cae4ae3b52a70fffabc170e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:45.504585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6cb9ce9cae4ae3b52a70fffabc170e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:45.505186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6cb9ce9cae4ae3b52a70fffabc170e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:45.584601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1153/2304] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:45 2022] DELETE /identity/v3/users/9a6cb9ce9cae4ae3b52a70fffabc170e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:45.588960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:45.589430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:45.589551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:45.608670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:45.609853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:45.666717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1152/2305] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:45.676365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:45.677780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:45.691753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:45.736519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3823b-e206-4477-9330-704d3fc5d2da admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:45.740130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f47bf-ec4d-419e-a31f-1b11cfd2197a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:45.740725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f47bf-ec4d-419e-a31f-1b11cfd2197a service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:45.741000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f47bf-ec4d-419e-a31f-1b11cfd2197a service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:45.741268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f47bf-ec4d-419e-a31f-1b11cfd2197a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:45.750543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f47bf-ec4d-419e-a31f-1b11cfd2197a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:45.752589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f47bf-ec4d-419e-a31f-1b11cfd2197a service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:45.761860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1154/2306] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:46.139937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.141380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:46.153639 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e597b-519f-45a1-9efb-ff087c710511 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.159821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.160468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.160863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.161145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] PATH_INFO: `/v3/projects/5d368aa1f0eb41569a799a30213635af` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.162322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d368aa1f0eb41569a799a30213635af)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:46.164243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:46.177075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:46.312874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d368aa1f0eb41569a799a30213635af', 'request_id': 'req-de5e2baa-0cc3-471f-93da-00ef76f49bd6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:46.425262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d368aa1f0eb41569a799a30213635af', 'request_id': 'req-de5e2baa-0cc3-471f-93da-00ef76f49bd6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:46.432911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d368aa1f0eb41569a799a30213635af', 'request_id': 'req-de5e2baa-0cc3-471f-93da-00ef76f49bd6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:46.445075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] The token cache is being invalidate because project 5d368aa1f0eb41569a799a30213635af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:46.445362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:46.446008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-de5e2baa-0cc3-471f-93da-00ef76f49bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:46.447536 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1153/2307] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:46 2022] DELETE /identity/v3/projects/5d368aa1f0eb41569a799a30213635af => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:46.452489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.452959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.453073 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.473278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:46.474514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:46.532405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1155/2308] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:46.540230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.540508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.540775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.563209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:46.564395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:46.572691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1154/2309] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:16:46.576848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.587009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8514834-06a3-4a42-a520-6fbc4e9b0075 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.591798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.592404 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.592686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.592960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.593601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:16:46.594855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:46.596532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:46.612228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1156/2310] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:16:46.613656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.623131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f5805-68ba-4e10-95fc-80849020c8f7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.627937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.628618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.628901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.629240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.629867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:46.631815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:46.729216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1155/2311] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:46 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:16:46.733516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.743415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f805a6-cfb7-43ac-b042-cffaacc131f7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.798537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.799160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.799437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.799729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.800361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:46.802258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:46.904388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1157/2312] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:46 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:46.909065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.919501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f6e8-13f4-4daf-9f78-6f43ea38b568 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.924329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.924917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.925222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.925527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.926199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:46.927828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:46.937799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1156/2313] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:46.941553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.951463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb86d8-22a2-4135-b276-6849b590e7f5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.956511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d66ffc-031e-48ec-9ac0-d0ccf5feee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.957090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d66ffc-031e-48ec-9ac0-d0ccf5feee76 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.957364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d66ffc-031e-48ec-9ac0-d0ccf5feee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.957644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d66ffc-031e-48ec-9ac0-d0ccf5feee76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:46.958319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d66ffc-031e-48ec-9ac0-d0ccf5feee76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:46.959959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d66ffc-031e-48ec-9ac0-d0ccf5feee76 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:46.970597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1158/2314] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:46.975401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:46.987831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba8517-8c5c-4346-9371-00798af273bd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:46.991211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:46.991852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:46.992129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:46.992402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] PATH_INFO: `/v3/projects/8a4584ce8ddb4b5d8612f9b47d6a7c07/users/97a9d00118ba4c3ea283f047f9d2e1c6/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.007627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4584ce8ddb4b5d8612f9b47d6a7c07, user_id=97a9d00118ba4c3ea283f047f9d2e1c6, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:47.017235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:47.082490 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1157/2315] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:46 2022] PUT /identity/v3/projects/8a4584ce8ddb4b5d8612f9b47d6a7c07/users/97a9d00118ba4c3ea283f047f9d2e1c6/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:47.093027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:47.093320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:47.093591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.120403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] MFA Rules not processed for user `97a9d00118ba4c3ea283f047f9d2e1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:47.121648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:47.130243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:47.186328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1159/2316] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:47.198460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:47.200279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:47.258073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862177-4cf5-4a3a-828d-ad0c1f56bd43 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:47.261688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0494a-3151-4773-91fd-26de0aed8623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:47.262283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0494a-3151-4773-91fd-26de0aed8623 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:47.262561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0494a-3151-4773-91fd-26de0aed8623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:47.262918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0494a-3151-4773-91fd-26de0aed8623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.271776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0494a-3151-4773-91fd-26de0aed8623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:47.273210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0494a-3151-4773-91fd-26de0aed8623 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:47.282039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1158/2317] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:47.305887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:47.373342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaf69b-5180-420c-a7a3-e3db645c11f5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:47.378622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:47.379319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:47.379655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:47.379860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] PATH_INFO: `/v3/users/97a9d00118ba4c3ea283f047f9d2e1c6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.381006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a9d00118ba4c3ea283f047f9d2e1c6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:47.382958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:47.540366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a9d00118ba4c3ea283f047f9d2e1c6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.552670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a9d00118ba4c3ea283f047f9d2e1c6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.559958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a9d00118ba4c3ea283f047f9d2e1c6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.560667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a9d00118ba4c3ea283f047f9d2e1c6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.561295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a9d00118ba4c3ea283f047f9d2e1c6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.635138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1160/2318] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:47 2022] DELETE /identity/v3/users/97a9d00118ba4c3ea283f047f9d2e1c6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:47.639842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:47.640001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:47.640423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:47.640423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.641376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:47.659255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:47.660461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:47.672935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:47.691697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:47.711638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27404280-242a-4971-80c7-d3dc487c1bae admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:47.716530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:47.717162 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:47.717444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:47.717715 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] PATH_INFO: `/v3/users/5b6d5a5a65d9463cb4a8e83f5573aa77` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.719138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6d5a5a65d9463cb4a8e83f5573aa77)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:47.721602 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:47.722265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1159/2319] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:16:47.732691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:47.734623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:47.790308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addf29d-47c7-4dcc-a47f-2f9167d3b006 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:47.793868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae559d-650d-42e2-8e27-9be9a4d2eb4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:47.794945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae559d-650d-42e2-8e27-9be9a4d2eb4c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:47.795223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae559d-650d-42e2-8e27-9be9a4d2eb4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:47.795492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae559d-650d-42e2-8e27-9be9a4d2eb4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:47.805177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae559d-650d-42e2-8e27-9be9a4d2eb4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:47.806601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae559d-650d-42e2-8e27-9be9a4d2eb4c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:47.816378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1160/2320] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:16:47.889138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6d5a5a65d9463cb4a8e83f5573aa77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.902127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6d5a5a65d9463cb4a8e83f5573aa77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.909504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6d5a5a65d9463cb4a8e83f5573aa77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.910363 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6d5a5a65d9463cb4a8e83f5573aa77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:47.910893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6d5a5a65d9463cb4a8e83f5573aa77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.007550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1161/2321] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:47 2022] DELETE /identity/v3/users/5b6d5a5a65d9463cb4a8e83f5573aa77 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:48.011897 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.012182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.012449 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.032692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:48.033887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.096102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1161/2322] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:48.105846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.107315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.165910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2f299-3fc1-408d-bd33-dfb29448b6c1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.169571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c77d5-ce40-4571-98bc-c91956141e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.170172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c77d5-ce40-4571-98bc-c91956141e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.170450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c77d5-ce40-4571-98bc-c91956141e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.170759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c77d5-ce40-4571-98bc-c91956141e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.180246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c77d5-ce40-4571-98bc-c91956141e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.181666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c77d5-ce40-4571-98bc-c91956141e84 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.191436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1162/2323] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:16:48.226211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.227721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.240931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f793a1-c0b3-4362-9a07-7d507566e2cd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.246094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.246798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.247243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.247522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] PATH_INFO: `/v3/projects/8a4584ce8ddb4b5d8612f9b47d6a7c07` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.248763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4584ce8ddb4b5d8612f9b47d6a7c07)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.250641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.349842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4584ce8ddb4b5d8612f9b47d6a7c07', 'request_id': 'req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.401657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4584ce8ddb4b5d8612f9b47d6a7c07', 'request_id': 'req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.408989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4584ce8ddb4b5d8612f9b47d6a7c07', 'request_id': 'req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.421149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] The token cache is being invalidate because project 8a4584ce8ddb4b5d8612f9b47d6a7c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:48.421578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.422270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6b54e20b-d37d-4341-b7ca-371a37c6c2f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.425321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1162/2324] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:48 2022] DELETE /identity/v3/projects/8a4584ce8ddb4b5d8612f9b47d6a7c07 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:48.433362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.433944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.434289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.455546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:48.456853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.504936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:48.520677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1163/2325] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:48.529765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.530051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.530321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.538812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:48.551746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:48.552949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.560899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1163/2326] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:48.565169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.574931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af8917-3cea-4b20-8535-8a018d1d750a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.579598 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.580248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.580619 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.581157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.581457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:16:48.581761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.583551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.586561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.587969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.596422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1164/2327] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:16:48.600557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.604463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178cb7b-a324-4e19-98ca-23206369d8e9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.609475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.610603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.610603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.610833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] PATH_INFO: `/v3/projects/4b1e369f3fb74aa5b1e7fd2d71468136` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.611249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f396c-d017-42e3-b671-39a9b52e91c3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.612421 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1e369f3fb74aa5b1e7fd2d71468136)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.614313 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.617069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.617659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.617937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.618194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.618958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.621265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.700339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1165/2328] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:48 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:16:48.703811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.742127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1e369f3fb74aa5b1e7fd2d71468136', 'request_id': 'req-3fec71ed-1ade-4ec9-8455-531ab07e00a2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.772641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e382efa-c212-4f03-a556-680e10502b36 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.823760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1e369f3fb74aa5b1e7fd2d71468136', 'request_id': 'req-3fec71ed-1ade-4ec9-8455-531ab07e00a2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.827706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.828318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.828767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.829042 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.829646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1e369f3fb74aa5b1e7fd2d71468136', 'request_id': 'req-3fec71ed-1ade-4ec9-8455-531ab07e00a2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.829749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.831650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.840612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] The token cache is being invalidate because project 4b1e369f3fb74aa5b1e7fd2d71468136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:48.840903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.841561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:48.843116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1164/2329] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:48 2022] DELETE /identity/v3/projects/4b1e369f3fb74aa5b1e7fd2d71468136 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:48.847957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.849233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.862143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec71ed-1ade-4ec9-8455-531ab07e00a2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.866828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.869405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.869405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.869405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] PATH_INFO: `/v3/users/a6abe88bd7634559848d36280713e14c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.869405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6abe88bd7634559848d36280713e14c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.871002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.921167 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1166/2330] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:48 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:16:48.925288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.926851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:48.938322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca209e7f-3258-45e9-b4e9-268bef4f8997 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.943100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.943675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.943949 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.944218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.944897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.946459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.956634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1167/2331] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:16:48.960452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:48.969879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b8c77-a9c9-4738-8144-d50de3f7a6cf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:48.975304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:48.975851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:48.976147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:48.976416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:48.977060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:48.978572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:48.988762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1168/2332] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:16:48.993175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:49.002917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a83fc3-9465-457d-ac78-2d1ecfff960b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:49.007636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d6fae-0833-40a0-880b-dd7d2597caef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:49.008301 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d6fae-0833-40a0-880b-dd7d2597caef admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:49.008585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d6fae-0833-40a0-880b-dd7d2597caef admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:49.008854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d6fae-0833-40a0-880b-dd7d2597caef admin admin] PATH_INFO: `/v3/projects/ea931246f8e74f6ab520b032fe2c5a3e/users/4d7de9e785544d14a901cd1c3e1d8bd0/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:49.023362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d6fae-0833-40a0-880b-dd7d2597caef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea931246f8e74f6ab520b032fe2c5a3e, user_id=4d7de9e785544d14a901cd1c3e1d8bd0, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:49.025988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d6fae-0833-40a0-880b-dd7d2597caef admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:49.077602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6abe88bd7634559848d36280713e14c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.093064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6abe88bd7634559848d36280713e14c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.100757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1169/2333] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:48 2022] PUT /identity/v3/projects/ea931246f8e74f6ab520b032fe2c5a3e/users/4d7de9e785544d14a901cd1c3e1d8bd0/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:16:49.101171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6abe88bd7634559848d36280713e14c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.101942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6abe88bd7634559848d36280713e14c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.102538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6abe88bd7634559848d36280713e14c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.110957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:49.111243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:49.111538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:49.124303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:49.139237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] MFA Rules not processed for user `4d7de9e785544d14a901cd1c3e1d8bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:49.140442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:49.178415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1165/2334] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:16:48 2022] DELETE /identity/v3/users/a6abe88bd7634559848d36280713e14c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:16:49.207034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1170/2335] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:16:49.220250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:49.224280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:49.285839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c178474-2b5d-49bb-b00e-0728cf3d208a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:49.289962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5b95-a445-4f9d-8b01-fc5f6f2891b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:49.290560 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5b95-a445-4f9d-8b01-fc5f6f2891b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:49.290869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5b95-a445-4f9d-8b01-fc5f6f2891b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:49.291137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5b95-a445-4f9d-8b01-fc5f6f2891b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:49.295281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:49.347403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc5b95-a445-4f9d-8b01-fc5f6f2891b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:49.348901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc5b95-a445-4f9d-8b01-fc5f6f2891b7 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:49.358842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1166/2336] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:49.634373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:49.636081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:49.707434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7ab7f-d6bf-4397-b785-c74f717b1803 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:49.766139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:49.766856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:49.767160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:49.769248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] PATH_INFO: `/v3/projects/9d217fc9950b4005a0c9b36efdd4685b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:49.769248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d217fc9950b4005a0c9b36efdd4685b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:49.770682 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:49.895126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:49.897979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d217fc9950b4005a0c9b36efdd4685b', 'request_id': 'req-2723beb9-04c1-43bf-9788-be487b9488ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.967077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d217fc9950b4005a0c9b36efdd4685b', 'request_id': 'req-2723beb9-04c1-43bf-9788-be487b9488ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.973919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d217fc9950b4005a0c9b36efdd4685b', 'request_id': 'req-2723beb9-04c1-43bf-9788-be487b9488ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.985060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] The token cache is being invalidate because project 9d217fc9950b4005a0c9b36efdd4685b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:16:49.985352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.986004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2723beb9-04c1-43bf-9788-be487b9488ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:16:49.988423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1171/2337] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:16:49 2022] DELETE /identity/v3/projects/9d217fc9950b4005a0c9b36efdd4685b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:49.993236 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:49.994926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:49.994926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.013123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:50.014322 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:50.075684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1167/2338] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:50.082750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.083134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.083467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.103900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:50.114535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:50.114535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1172/2339] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:50.118320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.128097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442012f-0d42-47df-9e3e-8fbdbc6ddba6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.132552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.133136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.133415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.133688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.134321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:16:50.134622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.136235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.147569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1168/2340] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:16:50.151637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.161572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dea590-4258-48fd-882c-ac3cd8018607 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.166226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.166848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.167121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.167387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.168033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.169909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.305767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1173/2341] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:16:50 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:50.309954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.319908 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab545d77-4cab-4aec-a47d-d575a7064f2a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.375685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.376276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.376557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.376831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.377440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.379438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.457890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1169/2342] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:16:50 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:16:50.461990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.473311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679a64e-d857-42fa-9381-eab22354772f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.477820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.478413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.478707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.478974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.479609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.481177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.491863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1174/2343] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:50.495801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.505766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef646-91bc-4c0f-a31a-491c0c2260ad admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.510345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1892c4-af0a-416f-9b0c-f8008a9d69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.510958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1892c4-af0a-416f-9b0c-f8008a9d69d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.511233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1892c4-af0a-416f-9b0c-f8008a9d69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.511500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1892c4-af0a-416f-9b0c-f8008a9d69d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.512128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1892c4-af0a-416f-9b0c-f8008a9d69d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.513729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1892c4-af0a-416f-9b0c-f8008a9d69d9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.524156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1170/2344] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:16:50.528479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.538980 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f460f82-5346-4626-8453-a0a4322e6dd1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.543498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.544139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.544419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.544693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] PATH_INFO: `/v3/projects/48dcf02947bd4522ac6886d8a0ca63ba/users/50cf4af54ca0488ebb6bb0f7c33d6d58/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.559599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48dcf02947bd4522ac6886d8a0ca63ba, user_id=50cf4af54ca0488ebb6bb0f7c33d6d58, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.562386 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.623964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1175/2345] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:16:50 2022] PUT /identity/v3/projects/48dcf02947bd4522ac6886d8a0ca63ba/users/50cf4af54ca0488ebb6bb0f7c33d6d58/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:16:50.629829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6d686-c0c6-4136-8ed0-c99fbdca0a95 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.630110 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6d686-c0c6-4136-8ed0-c99fbdca0a95 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.630380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6d686-c0c6-4136-8ed0-c99fbdca0a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.649376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3de6d686-c0c6-4136-8ed0-c99fbdca0a95 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:50.650561 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3de6d686-c0c6-4136-8ed0-c99fbdca0a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:50.655487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:50.707924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1171/2346] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:50.717535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:50.719608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:50.776243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9012698-a056-49e9-8407-40fec58bdaa4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:50.779752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ef830-bfdd-478c-908b-176e2d75b7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:50.780365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ef830-bfdd-478c-908b-176e2d75b7ab service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:50.780640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ef830-bfdd-478c-908b-176e2d75b7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:50.780909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ef830-bfdd-478c-908b-176e2d75b7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:50.789764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ef830-bfdd-478c-908b-176e2d75b7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:50.791232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ef830-bfdd-478c-908b-176e2d75b7ab service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:50.801252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1176/2347] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:51.070583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:51.080148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:51.080148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:51.096772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] MFA Rules not processed for user `4d7de9e785544d14a901cd1c3e1d8bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:51.098757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:51.159164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1172/2348] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:52.310844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef10a69-3ada-4725-8a49-8540009e034f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:52.311305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef10a69-3ada-4725-8a49-8540009e034f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:52.311628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef10a69-3ada-4725-8a49-8540009e034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:52.343333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6ef10a69-3ada-4725-8a49-8540009e034f None None] MFA Rules not processed for user `50cf4af54ca0488ebb6bb0f7c33d6d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:52.344823 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6ef10a69-3ada-4725-8a49-8540009e034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:52.424458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1177/2349] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:16:52.434356 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:52.435867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:52.496921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4903-c809-41ea-9656-2c39718bbd79 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:52.500619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb859a2-ff5a-4424-8ae0-11176cbb8fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:52.501337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb859a2-ff5a-4424-8ae0-11176cbb8fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:52.501795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb859a2-ff5a-4424-8ae0-11176cbb8fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:52.502072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb859a2-ff5a-4424-8ae0-11176cbb8fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:52.511853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb859a2-ff5a-4424-8ae0-11176cbb8fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:52.513389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb859a2-ff5a-4424-8ae0-11176cbb8fc5 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:52.522778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1173/2350] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:16:55.421519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:55.422044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:55.422140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:55.429339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:55.441763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] MFA Rules not processed for user `50cf4af54ca0488ebb6bb0f7c33d6d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:16:55.443067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:55.449910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1178/2351] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:16:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:02.518355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=73 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:17:05.572414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:17:19.732276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f46af-80d9-48b3-b3a5-d6766087c545 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:19.732785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f46af-80d9-48b3-b3a5-d6766087c545 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:19.732883 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f46af-80d9-48b3-b3a5-d6766087c545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:19.752310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-818f46af-80d9-48b3-b3a5-d6766087c545 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:19.753530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-818f46af-80d9-48b3-b3a5-d6766087c545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:19.811565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1174/2352] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:17:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:19.861377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:19.863020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:19.943747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5988c-4617-4d87-a5f9-ba978ddcf300 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:19.948712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32778e41-0c46-40b4-b0ae-c80be0799ea4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:19.949313 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32778e41-0c46-40b4-b0ae-c80be0799ea4 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:19.949586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32778e41-0c46-40b4-b0ae-c80be0799ea4 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:19.949852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32778e41-0c46-40b4-b0ae-c80be0799ea4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:19.959764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32778e41-0c46-40b4-b0ae-c80be0799ea4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:19.961373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32778e41-0c46-40b4-b0ae-c80be0799ea4 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:19.970569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1179/2353] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:17:20.607121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:20.607121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:20.607121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:20.624476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] MFA Rules not processed for user `22e92975af1d439f88e33cc494c88b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:20.625740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:20.692829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1175/2354] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:17:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:20.853431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79f672-f560-46e0-8687-1bc372ef150e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:20.853720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79f672-f560-46e0-8687-1bc372ef150e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:20.853999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79f672-f560-46e0-8687-1bc372ef150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:20.873758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0d79f672-f560-46e0-8687-1bc372ef150e None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:20.875016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0d79f672-f560-46e0-8687-1bc372ef150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:20.883311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1180/2355] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 10 23:17:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:20.897384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:20.909467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098eab-a4d0-413b-b6f7-86b7b211fa62 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:20.914711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:20.915316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:20.915597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:20.915869 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:20.924173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:20.925755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:20.935350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1176/2356] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:17:21.863181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0f6e-b56b-4add-a922-f24854a39848 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:21.863600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0f6e-b56b-4add-a922-f24854a39848 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:21.863890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0f6e-b56b-4add-a922-f24854a39848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:21.883389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-819f0f6e-b56b-4add-a922-f24854a39848 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:21.884570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-819f0f6e-b56b-4add-a922-f24854a39848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:21.891835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1181/2357] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:17:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:21.903893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:21.915942 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18583ddd-fdbb-43f4-b66a-01a423078d2a service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:21.921410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:21.922003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:21.922277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:21.922817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:21.932939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:21.934521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:21.942946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1177/2358] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:17:22.819467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ee949-42f8-4c7a-8d30-712db6e5a5f5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:22.831657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ee949-42f8-4c7a-8d30-712db6e5a5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:22.831657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ee949-42f8-4c7a-8d30-712db6e5a5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:22.842340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-930ee949-42f8-4c7a-8d30-712db6e5a5f5 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:22.843862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-930ee949-42f8-4c7a-8d30-712db6e5a5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:22.852252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1182/2359] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 10 23:17:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:22.866649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:22.877068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493b4a3-bedb-4524-be81-82c4fd508ff3 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:22.885381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:22.886280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:22.886791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:22.887230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:22.897818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:22.899446 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:22.907438 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1178/2360] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:17:23.683658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364415-aa3f-4e3d-8a83-b85af6150a5f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:23.684567 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364415-aa3f-4e3d-8a83-b85af6150a5f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:23.684997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364415-aa3f-4e3d-8a83-b85af6150a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:23.705948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-22364415-aa3f-4e3d-8a83-b85af6150a5f None None] MFA Rules not processed for user `50cf4af54ca0488ebb6bb0f7c33d6d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:23.707384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-22364415-aa3f-4e3d-8a83-b85af6150a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:23.714616 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1183/2361] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:17:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:25.926591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:25.928043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:25.941723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ae0f-a211-4006-98e8-f8023ce3be63 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:25.946180 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:25.946803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:25.947078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:25.947342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] PATH_INFO: `/v3/users/4d7de9e785544d14a901cd1c3e1d8bd0` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:25.948428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7de9e785544d14a901cd1c3e1d8bd0)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:25.950311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:26.118475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7de9e785544d14a901cd1c3e1d8bd0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:26.131730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7de9e785544d14a901cd1c3e1d8bd0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:26.133314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:17:26.139984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7de9e785544d14a901cd1c3e1d8bd0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:26.140772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7de9e785544d14a901cd1c3e1d8bd0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:26.141452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7de9e785544d14a901cd1c3e1d8bd0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:26.201106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1179/2362] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:17:25 2022] DELETE /identity/v3/users/4d7de9e785544d14a901cd1c3e1d8bd0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:17:26.205816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:26.206425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:26.206425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:26.227913 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:26.235422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:26.299604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1184/2363] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:26.312487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:26.314302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:26.391208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f840-dbe1-408e-8fdf-39a77c4b1fd2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:26.395666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251673c-a25f-4235-8470-ca273de6fcb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:26.397280 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251673c-a25f-4235-8470-ca273de6fcb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:26.397702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251673c-a25f-4235-8470-ca273de6fcb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:26.398146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251673c-a25f-4235-8470-ca273de6fcb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:26.425358 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251673c-a25f-4235-8470-ca273de6fcb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:26.426940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251673c-a25f-4235-8470-ca273de6fcb3 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:26.440255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1180/2364] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:17:26.746679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:26.746679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:26.762581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfa06a-0faa-4f6b-a427-b632b0b79de3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:26.769373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:26.770268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:26.770871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:26.771335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] PATH_INFO: `/v3/projects/ea931246f8e74f6ab520b032fe2c5a3e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:26.773591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea931246f8e74f6ab520b032fe2c5a3e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:26.779019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:26.941218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea931246f8e74f6ab520b032fe2c5a3e', 'request_id': 'req-deefa11a-8660-4c33-9e9c-402ca57775b9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:27.080689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea931246f8e74f6ab520b032fe2c5a3e', 'request_id': 'req-deefa11a-8660-4c33-9e9c-402ca57775b9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:27.083544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:17:27.087914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea931246f8e74f6ab520b032fe2c5a3e', 'request_id': 'req-deefa11a-8660-4c33-9e9c-402ca57775b9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:27.101819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] The token cache is being invalidate because project ea931246f8e74f6ab520b032fe2c5a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:17:27.102215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:27.102997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:17:27.111400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1185/2365] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:17:26 2022] DELETE /identity/v3/projects/ea931246f8e74f6ab520b032fe2c5a3e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:17:32.444565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536933-a6dc-419e-9994-fc539284916c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:32.445063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536933-a6dc-419e-9994-fc539284916c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:32.445176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536933-a6dc-419e-9994-fc539284916c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:32.464808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0c536933-a6dc-419e-9994-fc539284916c None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:32.468190 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0c536933-a6dc-419e-9994-fc539284916c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:32.522222 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1181/2366] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:17:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:17:32.535692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:32.537692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:32.616347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefa11a-8660-4c33-9e9c-402ca57775b9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:32.622754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713991da-dd92-4f64-93df-a2de9e4050fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:32.627439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713991da-dd92-4f64-93df-a2de9e4050fa service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:32.627439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713991da-dd92-4f64-93df-a2de9e4050fa service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:32.627439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713991da-dd92-4f64-93df-a2de9e4050fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:32.633794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713991da-dd92-4f64-93df-a2de9e4050fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:32.635563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713991da-dd92-4f64-93df-a2de9e4050fa service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:32.654416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1186/2367] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:17:42.515646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=57,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:17:42.641164 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=37,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:17:52.454355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf8182-ab9e-4d86-b0ac-bcdef5cb0a90 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:52.454949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf8182-ab9e-4d86-b0ac-bcdef5cb0a90 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:52.455065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf8182-ab9e-4d86-b0ac-bcdef5cb0a90 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:52.456416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1182/2368] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 23:17:52.463173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cc0e-1199-412c-806a-ad91210461ad None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:52.463722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cc0e-1199-412c-806a-ad91210461ad None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:52.463862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249cc0e-1199-412c-806a-ad91210461ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:52.488126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3249cc0e-1199-412c-806a-ad91210461ad None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:52.489423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3249cc0e-1199-412c-806a-ad91210461ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:52.554311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1187/2369] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:17:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:17:52.565729 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf8182-ab9e-4d86-b0ac-bcdef5cb0a90 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:52.567349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-35cf8182-ab9e-4d86-b0ac-bcdef5cb0a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:52.624204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf8182-ab9e-4d86-b0ac-bcdef5cb0a90 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:52.627839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3694ae8-a05b-46ed-b9c8-a954cbc11956 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:52.628425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3694ae8-a05b-46ed-b9c8-a954cbc11956 service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:52.628702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3694ae8-a05b-46ed-b9c8-a954cbc11956 service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:52.628970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3694ae8-a05b-46ed-b9c8-a954cbc11956 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:52.639441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3694ae8-a05b-46ed-b9c8-a954cbc11956 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:52.640862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3694ae8-a05b-46ed-b9c8-a954cbc11956 service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:52.651288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1183/2370] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:17:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:17:52.847416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1b9dc-aa76-4ff6-beba-fc85e8d574d1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:52.847628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1b9dc-aa76-4ff6-beba-fc85e8d574d1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:52.847888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1b9dc-aa76-4ff6-beba-fc85e8d574d1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:52.849240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1188/2371] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:17:52.853803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e80c2-0945-4454-a7e9-2c46576204d6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:52.854089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e80c2-0945-4454-a7e9-2c46576204d6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:52.854359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e80c2-0945-4454-a7e9-2c46576204d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:52.874544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-be1e80c2-0945-4454-a7e9-2c46576204d6 None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:52.875764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-be1e80c2-0945-4454-a7e9-2c46576204d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:52.926743 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1184/2372] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:17:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:52.939181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1b9dc-aa76-4ff6-beba-fc85e8d574d1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:52.940893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-49c1b9dc-aa76-4ff6-beba-fc85e8d574d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:52.953944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1b9dc-aa76-4ff6-beba-fc85e8d574d1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:52.957423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd256a1-6afd-4610-b2c5-64d8c85879b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:52.958010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd256a1-6afd-4610-b2c5-64d8c85879b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:52.958290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd256a1-6afd-4610-b2c5-64d8c85879b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:52.958566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd256a1-6afd-4610-b2c5-64d8c85879b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:52.966894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd256a1-6afd-4610-b2c5-64d8c85879b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:52.968591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd256a1-6afd-4610-b2c5-64d8c85879b6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:52.977039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1189/2373] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:17:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:17:53.102177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294770a-2e40-4f22-bcb3-a7978b7d9ee2 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:53.102405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294770a-2e40-4f22-bcb3-a7978b7d9ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:53.102708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294770a-2e40-4f22-bcb3-a7978b7d9ee2 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:53.104051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1185/2374] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:17:53.109827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e6315-7b27-4252-b3eb-191d5187278c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:53.110184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e6315-7b27-4252-b3eb-191d5187278c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:53.110463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e6315-7b27-4252-b3eb-191d5187278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:53.130467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-773e6315-7b27-4252-b3eb-191d5187278c None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:53.131758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-773e6315-7b27-4252-b3eb-191d5187278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:53.140882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1190/2375] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:17:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:53.151367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294770a-2e40-4f22-bcb3-a7978b7d9ee2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:53.153121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1294770a-2e40-4f22-bcb3-a7978b7d9ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:53.166523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294770a-2e40-4f22-bcb3-a7978b7d9ee2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:53.169929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:53.175068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:53.175068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:53.175068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:53.181291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:53.182746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:53.193275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1186/2376] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:17:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:17:57.179290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d895f1f-6c53-4169-b149-3e702ce08da9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:57.179757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d895f1f-6c53-4169-b149-3e702ce08da9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:57.180012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d895f1f-6c53-4169-b149-3e702ce08da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:57.205428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1d895f1f-6c53-4169-b149-3e702ce08da9 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:57.206631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d895f1f-6c53-4169-b149-3e702ce08da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:57.215305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1191/2377] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:17:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:57.228872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:57.240760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d273-dbf6-49b2-afbf-3c824ce7bbab service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:57.246699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:57.247321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:57.247597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:57.247861 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:57.256213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:57.257895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:57.267078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1187/2378] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:17:59.454394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032afedf-e45c-4354-90d2-95365e857405 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:59.454976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032afedf-e45c-4354-90d2-95365e857405 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:59.454976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032afedf-e45c-4354-90d2-95365e857405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:59.478008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-032afedf-e45c-4354-90d2-95365e857405 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:17:59.479319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-032afedf-e45c-4354-90d2-95365e857405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:17:59.490023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1192/2379] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:17:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:17:59.507665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:17:59.521297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d940b6-4cc9-45c1-a712-979cb8819c29 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:17:59.527592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:17:59.528435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:17:59.528901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:17:59.529371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:17:59.539958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:17:59.541881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:17:59.552208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1188/2380] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:18:09.470790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=46 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:09.549038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=67 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:14.633351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2324b9-acd7-4baf-8f85-93927443384f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:14.633867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2324b9-acd7-4baf-8f85-93927443384f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:14.633966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2324b9-acd7-4baf-8f85-93927443384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:14.652859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-de2324b9-acd7-4baf-8f85-93927443384f None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:14.654055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-de2324b9-acd7-4baf-8f85-93927443384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:14.661142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1193/2381] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 10 23:18:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:14.672984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:14.683469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86734ab-8011-49c1-9a64-7b553cce4544 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:14.688514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871b756-b327-4671-a5a9-f70265ac5276 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:14.689094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871b756-b327-4671-a5a9-f70265ac5276 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:14.689376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871b756-b327-4671-a5a9-f70265ac5276 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:14.689650 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871b756-b327-4671-a5a9-f70265ac5276 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:14.698258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871b756-b327-4671-a5a9-f70265ac5276 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:14.699867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871b756-b327-4671-a5a9-f70265ac5276 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:14.707929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1189/2382] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:18:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:18:18.079391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b8731-0d1c-458d-82f4-f30beead15e3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:18.079927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b8731-0d1c-458d-82f4-f30beead15e3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:18.079927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b8731-0d1c-458d-82f4-f30beead15e3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:18.081259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1194/2383] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:18:18.107607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:18.108109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:18.108296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:18.128182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] MFA Rules not processed for user `2604f58c168e432d812608a2351610bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:18.129399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:18.194347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1190/2384] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:18:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:18.207424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b8731-0d1c-458d-82f4-f30beead15e3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:18.209462 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f21b8731-0d1c-458d-82f4-f30beead15e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:18.215044 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:18.216491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:18.231422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b8731-0d1c-458d-82f4-f30beead15e3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:18.236612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:18.237258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:18.237479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:18.237745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:18.247724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:18.249318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:18.285001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1195/2385] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:18:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:18:18.301576 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd01d1-d320-4c77-ab5d-715bf1d2e734 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:18.306240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:18.306923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:18.307205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:18.307477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] PATH_INFO: `/v3/users/50cf4af54ca0488ebb6bb0f7c33d6d58` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:18.308640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cf4af54ca0488ebb6bb0f7c33d6d58)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:18.310644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:18.419524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cf4af54ca0488ebb6bb0f7c33d6d58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:18.435378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cf4af54ca0488ebb6bb0f7c33d6d58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:18.442602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cf4af54ca0488ebb6bb0f7c33d6d58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:18.443419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cf4af54ca0488ebb6bb0f7c33d6d58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:18.444049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-338d1dd2-7fb4-495b-b889-1abd8de047c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cf4af54ca0488ebb6bb0f7c33d6d58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:18.526611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1191/2386] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:18:18 2022] DELETE /identity/v3/users/50cf4af54ca0488ebb6bb0f7c33d6d58 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:18:18.796489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:18.807552 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e10f4-9619-4267-a8f7-3e98e47a7bb6 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:18.864704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:18.865331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:18.865637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:18.865908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] PATH_INFO: `/v3/projects/48dcf02947bd4522ac6886d8a0ca63ba` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:18.867038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48dcf02947bd4522ac6886d8a0ca63ba)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:18.868925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:18.978025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48dcf02947bd4522ac6886d8a0ca63ba', 'request_id': 'req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:19.039659 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48dcf02947bd4522ac6886d8a0ca63ba', 'request_id': 'req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:19.045519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48dcf02947bd4522ac6886d8a0ca63ba', 'request_id': 'req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:19.056447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] The token cache is being invalidate because project 48dcf02947bd4522ac6886d8a0ca63ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:18:19.056734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:19.057364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ae4bdaa6-9aab-4ec8-9767-3b6664c95b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:18:19.058897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1196/2387] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:18:18 2022] DELETE /identity/v3/projects/48dcf02947bd4522ac6886d8a0ca63ba => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:18:19.064245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.064528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.064805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.072402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:19.084933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:19.086137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:19.144899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1192/2388] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:19.153198 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.153645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.155006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.173013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:19.174223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:19.182221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1197/2389] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:18:19.186465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.197328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094681ea-53b8-4e9e-8eb7-6e4bbac4278a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.202270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.202905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.203176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.203443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.204132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:18:19.204430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.206010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.218451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1193/2390] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:18:19.222688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.232096 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c7130-bb06-46f5-b85c-ce42682253de None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.237945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.238744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.239100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.239370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.240110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.242048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.339340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1198/2391] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:18:19 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:18:19.343906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.354264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2eb2-1720-4308-b740-c580244b4e99 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.409140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.409731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.410004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.410270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.410906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.412875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.480076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1194/2392] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:18:19 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:18:19.484136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.495043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a41c0-05dd-40de-a99c-8d1decd5e0fa admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.499925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.500524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.500803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.501072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.501710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.503347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.513372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1199/2393] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:18:19.517431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.527969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e785e-b314-417d-a1c5-c03d2063785e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.532570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145057c-d3e0-44e8-96f4-d6f7f4a50a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.533158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145057c-d3e0-44e8-96f4-d6f7f4a50a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.533437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145057c-d3e0-44e8-96f4-d6f7f4a50a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.533709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145057c-d3e0-44e8-96f4-d6f7f4a50a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.534344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145057c-d3e0-44e8-96f4-d6f7f4a50a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.535980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145057c-d3e0-44e8-96f4-d6f7f4a50a6b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.546994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1195/2394] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:18:19.551075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.560987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053afd42-e431-435f-93b7-26d68afe889b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.565531 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.566171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.566452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.566753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] PATH_INFO: `/v3/projects/e855cc11f6a142f9a6d31088812beeef/users/d0a56142f0fc4c32b7e34ec74803407f/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.580903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e855cc11f6a142f9a6d31088812beeef, user_id=d0a56142f0fc4c32b7e34ec74803407f, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.583548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.652376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1200/2395] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:18:19 2022] PUT /identity/v3/projects/e855cc11f6a142f9a6d31088812beeef/users/d0a56142f0fc4c32b7e34ec74803407f/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:18:19.657943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.658230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.658503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.679282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:19.683010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:19.743220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1196/2396] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:18:19.753336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:19.755150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:19.765930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:19.816022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a9b7b-12e0-4cc2-a731-176f2c6adfa9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:19.820095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:19.820692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:19.820966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:19.821232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:19.831151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:19.832519 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:19.843151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1201/2397] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:18:21.600692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:21.602440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:21.614690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:21.665548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cf847-f84f-488d-8746-cc94982f4626 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:21.669127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:21.669731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:21.670005 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:21.670274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:21.672077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:21.749469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:21.751092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd82a89-f293-4a29-8531-27e239d5726b service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:21.764427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1197/2398] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:18:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:18:23.239621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:23.241793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:23.253365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7c47-395b-4034-a64f-beb967246394 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:23.257763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:23.258323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:23.258680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:23.258964 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:23.260785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:23.276114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:23.277586 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:23.291883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1202/2399] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:18:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:18:25.265937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c3e76-60d8-4aaa-ba0f-e69fb286ce2c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:25.266477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c3e76-60d8-4aaa-ba0f-e69fb286ce2c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:25.266477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c3e76-60d8-4aaa-ba0f-e69fb286ce2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:25.294637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-186c3e76-60d8-4aaa-ba0f-e69fb286ce2c None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:25.298859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-186c3e76-60d8-4aaa-ba0f-e69fb286ce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:25.384050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1198/2400] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 10 23:18:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:25.411120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:25.415966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:25.531854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0750eb6-c566-4831-8eea-dd347c9be497 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:25.539844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:25.540566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:25.540957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:25.541343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:25.551231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:25.553213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:25.567444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1203/2401] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:18:25.647150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:25.647150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:25.647150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:25.672032 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] MFA Rules not processed for user `d0a56142f0fc4c32b7e34ec74803407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:25.673325 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:25.734405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:25.769670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1199/2402] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:25.774309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:25.791112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783883c-cb6b-47bf-85b3-9c4d1f98a7ef service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:25.797514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:25.798129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:25.798645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:25.799131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:25.799994 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:25.801912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:25.932396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1204/2403] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:18:25 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:18:25.935888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:25.955910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9971c61-4f7b-4822-99fe-25b472221407 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:26.040901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:26.041838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:26.042208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:26.042540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:26.043328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:26.045465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:26.206274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1200/2404] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:18:25 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:18:26.210728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:26.221113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d8e9c-c26d-4930-a957-99e79daf3727 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:26.225671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:26.226265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:26.226574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:26.226858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:26.227491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:26.229081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:26.239661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1205/2405] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:18:26.248839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:26.260058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01035569-ad6c-4e97-9bde-748e4886703b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:26.264936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:26.265603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:26.265880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:26.266148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] PATH_INFO: `/v3/projects/3cf92ba7f2944bb89fafda9d273eedac/users/d5903cbd5bb646a7acf4c5a553cbe1f5/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:26.281990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf92ba7f2944bb89fafda9d273eedac, user_id=d5903cbd5bb646a7acf4c5a553cbe1f5, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:26.284668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:26.446623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1201/2406] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:18:26 2022] PUT /identity/v3/projects/3cf92ba7f2944bb89fafda9d273eedac/users/d5903cbd5bb646a7acf4c5a553cbe1f5/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:18:26.450680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:26.460972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7cad-6114-419c-aa97-9a62cfefc6c6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:26.512271 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:26.522350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:26.523010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:26.523293 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:26.523569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:26.524215 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:26.525848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:26.538147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1206/2407] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:18:26.542223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:26.552453 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23717e0b-0b6b-47a3-9190-7f99ffa092f9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:26.557602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78770b-a427-447d-ad3f-be7c33fdb723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:26.558241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78770b-a427-447d-ad3f-be7c33fdb723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:26.558520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78770b-a427-447d-ad3f-be7c33fdb723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:26.558848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78770b-a427-447d-ad3f-be7c33fdb723 admin admin] PATH_INFO: `/v3/projects/3cf92ba7f2944bb89fafda9d273eedac/users/d5903cbd5bb646a7acf4c5a553cbe1f5/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:26.560877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78770b-a427-447d-ad3f-be7c33fdb723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf92ba7f2944bb89fafda9d273eedac, user_id=d5903cbd5bb646a7acf4c5a553cbe1f5, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:26.563642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78770b-a427-447d-ad3f-be7c33fdb723 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:26.574989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1202/2408] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:18:26 2022] PUT /identity/v3/projects/3cf92ba7f2944bb89fafda9d273eedac/users/d5903cbd5bb646a7acf4c5a553cbe1f5/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:18:31.106192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:31.186442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fd858-f958-4520-997a-16bfa9158796 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:31.247683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:31.250336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:31.250336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:31.250336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:31.251234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:31.333238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:31.334699 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:31.346553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1207/2409] 10.222.0.78 () {60 vars in 1396 bytes} [Wed Aug 10 23:18:31 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:18:32.835891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a650c75-48ad-4fc1-be53-b407de497e6a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:32.836465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a650c75-48ad-4fc1-be53-b407de497e6a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:32.836465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a650c75-48ad-4fc1-be53-b407de497e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:32.866196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7a650c75-48ad-4fc1-be53-b407de497e6a None None] MFA Rules not processed for user `d5903cbd5bb646a7acf4c5a553cbe1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:32.867504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7a650c75-48ad-4fc1-be53-b407de497e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:32.942948 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1203/2410] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:32.956738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:32.958154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:33.019542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:33.036225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15653728-3a40-4997-a455-ff0c8c78da6a service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:33.043684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d76b7-904c-43a1-9057-9b236599e658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:33.044135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d76b7-904c-43a1-9057-9b236599e658 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:33.044494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d76b7-904c-43a1-9057-9b236599e658 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:33.045086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d76b7-904c-43a1-9057-9b236599e658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:33.125017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d76b7-904c-43a1-9057-9b236599e658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:33.126642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d76b7-904c-43a1-9057-9b236599e658 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:33.137348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1208/2411] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:18:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:18:36.259485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:36.260019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:36.260142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:36.290964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] MFA Rules not processed for user `d0a56142f0fc4c32b7e34ec74803407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:36.292513 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:36.300593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1204/2412] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:18:43.132647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=29 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:44.654922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6511c14-708b-4c72-b166-84029313365d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:44.655628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6511c14-708b-4c72-b166-84029313365d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:44.655628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6511c14-708b-4c72-b166-84029313365d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:44.674428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d6511c14-708b-4c72-b166-84029313365d None None] MFA Rules not processed for user `ffb46379ef204df6bebd58f3bdb78ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:18:44.675749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d6511c14-708b-4c72-b166-84029313365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:44.683298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1209/2413] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:18:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:18:46.281745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:18:51.344267 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:18:51.354844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da254-2c83-4741-8d74-d3d55a13321a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:18:51.358595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:18:51.359258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:18:51.360405 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:18:51.360566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:18:51.362452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:18:51.373021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:18:51.374511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:18:51.382478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1205/2414] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:18:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:18:54.669284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:00.767772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6511c14-708b-4c72-b166-84029313365d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:00.777449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6511c14-708b-4c72-b166-84029313365d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:00.781310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:00.781911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:00.782188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:00.782455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:00.784230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:00.793690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:00.795142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:00.803046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1210/2415] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:19:01.378891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:01.578593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:01.580093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:01.652364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c19902-f48c-4c46-8499-69e55fd29636 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:01.657330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:01.657937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:01.658216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:01.658484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:01.660253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:01.673291 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:01.675105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:01.685384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1206/2416] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:19:05.647772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:05.661403 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbea6b-be95-452e-9a7b-b947a35c3963 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:05.667455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:05.668053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:05.668344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:05.668599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:05.670468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:05.683414 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:05.685072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:05.699533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1211/2417] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:19:11.681258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=32 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:15.692300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=12 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:19.578296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:19.588328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d7bab-1a11-464e-ba79-67c3eb4ae712 service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:19.592131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:19.592711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:19.592987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:19.593253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:19.602715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:19.604152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:19.613373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1207/2418] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:19:22.635475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:22.646858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad4f4d-e6ca-4453-9961-ed9507550d55 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:22.652325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:22.652911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:22.653189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:22.653530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:22.655298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:22.717974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:22.719573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb029ad8-272b-482b-8583-8ad64204b858 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:22.728475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1212/2419] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:19:29.609306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:32.724255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=28 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:35.355307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:35.356735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:35.372683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1113b-c520-49ba-b315-b4226fcabaac service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:35.378137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:35.378747 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:35.379025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:35.379295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:35.380963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:35.450220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:35.451851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:35.461148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1208/2420] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:19:38.429374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7e6c-cc4d-42f3-ad3d-54a44c8e2d71 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:38.429894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7e6c-cc4d-42f3-ad3d-54a44c8e2d71 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:38.429993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7e6c-cc4d-42f3-ad3d-54a44c8e2d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:38.449775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bbac7e6c-cc4d-42f3-ad3d-54a44c8e2d71 None None] MFA Rules not processed for user `22e92975af1d439f88e33cc494c88b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:19:38.451010 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bbac7e6c-cc4d-42f3-ad3d-54a44c8e2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:38.457928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1213/2421] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:19:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:19:38.844217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:38.853606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b739e3-f3a6-4fc7-92bb-ff052c349680 service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:38.857232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:38.857813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:38.858091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:38.858364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:38.860114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:38.869888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:38.871350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:38.879169 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1209/2422] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:19:46.332492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:46.334248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:46.334248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:46.352460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] MFA Rules not processed for user `45dd5b35a039468d99ec0d393056c5af`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:19:46.354351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:46.361411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1214/2423] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:19:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:19:48.875719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=34 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:50.584431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:50.598886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb523a97-0bad-4b74-83dd-99855a56b778 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:50.604605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:50.608251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:50.608251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:50.608251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:50.608251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:50.620381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:50.622050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:50.646550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1210/2424] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:19:56.347227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:19:58.162430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:19:58.179627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568c8d8-166b-40a5-a73e-50dbcbf9126b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:19:58.186204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:19:58.187118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:19:58.187605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:19:58.188141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:19:58.190043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:19:58.203431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:19:58.204935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:19:58.218396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1215/2425] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:19:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:20:00.628608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:20:08.212950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:20:14.132667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:14.141919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5b54a-b970-4dd0-9f90-ae627afba6fd service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:14.145461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:14.146037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:14.146315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:14.146589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:14.148335 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:14.209269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:14.210725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:14.220902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1211/2426] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:20:17.888777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:17.890202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:17.905711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c85cb-53e2-4c5d-a08c-52b797710e71 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:17.913499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:17.914106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:17.914391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:17.914681 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:17.916496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:17.930104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:17.931790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:17.942974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1216/2427] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:20:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:20:18.011946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:18.014840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:18.027703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201a127-162f-400b-83aa-79eaa504379c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:18.032713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:18.033461 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:18.033812 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:18.034177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] PATH_INFO: `/v3/users/ffb46379ef204df6bebd58f3bdb78ff6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:18.035409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb46379ef204df6bebd58f3bdb78ff6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:18.037464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:18.160324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb46379ef204df6bebd58f3bdb78ff6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.172668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb46379ef204df6bebd58f3bdb78ff6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.179479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb46379ef204df6bebd58f3bdb78ff6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.180159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb46379ef204df6bebd58f3bdb78ff6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.180757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f5a6888c-cf02-4b64-866b-64ada4f5224e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb46379ef204df6bebd58f3bdb78ff6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.232554 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1212/2428] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:20:18 2022] DELETE /identity/v3/users/ffb46379ef204df6bebd58f3bdb78ff6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:20:18.451516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:18.464294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790281-a9c1-4fb9-93f0-9a72fe2815e3 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:18.521845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:18.522468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:18.522782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:18.523061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] PATH_INFO: `/v3/projects/be0d52e189904e3c9971960d3ff4f58b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:18.524232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0d52e189904e3c9971960d3ff4f58b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:18.526128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:18.638737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0d52e189904e3c9971960d3ff4f58b', 'request_id': 'req-fa57f1d4-7c70-4576-8024-233cb99e4fd1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.706480 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0d52e189904e3c9971960d3ff4f58b', 'request_id': 'req-fa57f1d4-7c70-4576-8024-233cb99e4fd1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.712150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0d52e189904e3c9971960d3ff4f58b', 'request_id': 'req-fa57f1d4-7c70-4576-8024-233cb99e4fd1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.723956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] The token cache is being invalidate because project be0d52e189904e3c9971960d3ff4f58b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:20:18.724245 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.724883 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa57f1d4-7c70-4576-8024-233cb99e4fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:20:18.726340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1217/2429] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:20:18 2022] DELETE /identity/v3/projects/be0d52e189904e3c9971960d3ff4f58b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:20:18.732017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be104be-4aed-4437-b781-185760c5c047 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:18.732310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be104be-4aed-4437-b781-185760c5c047 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:18.732589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be104be-4aed-4437-b781-185760c5c047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:18.751943 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8be104be-4aed-4437-b781-185760c5c047 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:20:18.753173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8be104be-4aed-4437-b781-185760c5c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:18.812059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1213/2430] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:20:18.819738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:18.820020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:18.820289 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:18.840094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:20:18.841549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:18.849532 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1218/2431] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:20:18.853742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be104be-4aed-4437-b781-185760c5c047 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:18.863950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be104be-4aed-4437-b781-185760c5c047 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:18.868432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:18.869011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:18.869285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:18.869548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:18.870170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:20:18.870462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:18.872125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:18.873520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:20:18.883626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1214/2432] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:20:18.887810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:18.897187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92c1f-e631-47c2-86b3-8f558c6d1682 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:18.903360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:18.903360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:18.903661 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:18.903837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:18.904449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:18.906347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:18.965265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1219/2433] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:20:18 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:20:18.969774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:18.979871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e978f6-9055-4555-b82f-6bd5df2d0259 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:19.034710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:19.035373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:19.035569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:19.035842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:19.036452 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:19.038364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:19.130206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1215/2434] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:20:18 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:20:19.139985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:19.150319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae251521-20ad-408e-b44f-553ae7d8dd82 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:19.155604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:19.156202 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:19.156473 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:19.156740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:19.157368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:19.159001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:19.169440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1220/2435] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:20:19.173530 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:19.188692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3f6cb-971c-41f9-9dbc-a10506e48a3a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:19.193336 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734a341-7d67-430d-b71a-3053f57c896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:19.193926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a341-7d67-430d-b71a-3053f57c896a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:19.194204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a341-7d67-430d-b71a-3053f57c896a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:19.194477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a341-7d67-430d-b71a-3053f57c896a admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:19.195152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734a341-7d67-430d-b71a-3053f57c896a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:19.196730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734a341-7d67-430d-b71a-3053f57c896a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:19.207676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1216/2436] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:20:19.212048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:19.222850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c050904-193e-4fa2-8a7f-337bad5ad467 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:19.227332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:19.230124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:19.230124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:19.230124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] PATH_INFO: `/v3/projects/e72afdc971a44dd6a4eaabd40c40bbcb/users/35af5d49420a437193f2525bda7550d9/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:19.242847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e72afdc971a44dd6a4eaabd40c40bbcb, user_id=35af5d49420a437193f2525bda7550d9, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:19.245505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:19.416254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1221/2437] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:20:19 2022] PUT /identity/v3/projects/e72afdc971a44dd6a4eaabd40c40bbcb/users/35af5d49420a437193f2525bda7550d9/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:20:19.421407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2798b-125d-4a3d-a7a3-2235dd712489 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:19.421694 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2798b-125d-4a3d-a7a3-2235dd712489 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:19.421970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2798b-125d-4a3d-a7a3-2235dd712489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:19.442307 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-44c2798b-125d-4a3d-a7a3-2235dd712489 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:20:19.443574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-44c2798b-125d-4a3d-a7a3-2235dd712489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:19.502170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1217/2438] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:20:19.512475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:19.514185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:19.528160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:20:19.569312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91912b6-c1ea-4c4a-9f52-3a9034f846fd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:19.572924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:19.574154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:19.574434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:19.574719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:19.583601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:19.585051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:19.595176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1222/2439] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:20:24.971841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4315-3a62-4a67-9d11-cb5abe2500fe None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:24.972334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4315-3a62-4a67-9d11-cb5abe2500fe None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:24.972445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4315-3a62-4a67-9d11-cb5abe2500fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:25.003272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f41a4315-3a62-4a67-9d11-cb5abe2500fe None None] MFA Rules not processed for user `35af5d49420a437193f2525bda7550d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:20:25.004506 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f41a4315-3a62-4a67-9d11-cb5abe2500fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:25.081364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1218/2440] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:20:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:20:25.097870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:20:25.099455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:25.181669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d4fc-a355-46f7-85d9-0d93ee6d3fb3 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:20:25.187530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89bbfb-48bd-4360-abd8-2b79c0592b8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:20:25.188155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89bbfb-48bd-4360-abd8-2b79c0592b8a service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:25.188430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89bbfb-48bd-4360-abd8-2b79c0592b8a service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:25.188698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89bbfb-48bd-4360-abd8-2b79c0592b8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:25.197735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89bbfb-48bd-4360-abd8-2b79c0592b8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:20:25.200058 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89bbfb-48bd-4360-abd8-2b79c0592b8a service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:20:25.209697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1223/2441] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:20:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:20:27.599775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:20:27.600864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:20:27.601054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:20:27.608890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:20:27.635593 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] MFA Rules not processed for user `35af5d49420a437193f2525bda7550d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:20:27.636833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:20:27.652255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1219/2442] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:20:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:20:35.205316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=54 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:20:37.809478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:21:32.668442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f249be-eaa8-4952-bd55-08fc8c713eea None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:21:32.669253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f249be-eaa8-4952-bd55-08fc8c713eea None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:21:32.669436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f249be-eaa8-4952-bd55-08fc8c713eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:21:32.697153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e4f249be-eaa8-4952-bd55-08fc8c713eea None None] MFA Rules not processed for user `22e92975af1d439f88e33cc494c88b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:21:32.698260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e4f249be-eaa8-4952-bd55-08fc8c713eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:21:32.767143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1224/2443] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:21:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:21:34.797177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:21:34.798583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:21:34.934571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e4014-9609-4f40-951e-c21baa4d815b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:21:34.939692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:21:34.940281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:21:34.940552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:21:34.940821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:21:34.942546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:21:34.954367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:21:34.955966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:21:34.966466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1220/2444] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:21:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:21:42.765053 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=37 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:21:43.555274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9d6b1-1d0b-4800-b699-e146fd178145 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:21:43.555605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9d6b1-1d0b-4800-b699-e146fd178145 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:21:43.556419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9d6b1-1d0b-4800-b699-e146fd178145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:21:43.586338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ace9d6b1-1d0b-4800-b699-e146fd178145 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:21:43.588195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ace9d6b1-1d0b-4800-b699-e146fd178145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:21:43.656813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1225/2445] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:21:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:21:43.819258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:21:43.820582 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:21:43.898543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d27754-8f7c-4fe7-8d59-8f9b6085713f service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:21:43.904235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc6f68-967c-4a71-bbeb-9eb07bece3df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:21:43.904840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc6f68-967c-4a71-bbeb-9eb07bece3df service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:21:43.905116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc6f68-967c-4a71-bbeb-9eb07bece3df service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:21:43.905383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc6f68-967c-4a71-bbeb-9eb07bece3df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:21:43.913940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc6f68-967c-4a71-bbeb-9eb07bece3df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:21:43.915568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc6f68-967c-4a71-bbeb-9eb07bece3df service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:21:43.926518 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1221/2446] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:21:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:21:53.649440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=32 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:21:53.921175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:22:02.493246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c49262-3f68-4d54-b6bd-2424456fd3d6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:02.493798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c49262-3f68-4d54-b6bd-2424456fd3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:02.493798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c49262-3f68-4d54-b6bd-2424456fd3d6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:02.496408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1226/2447] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:22:02.501750 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92e244-056c-41be-a0c7-c6225a1d56cd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:02.502223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92e244-056c-41be-a0c7-c6225a1d56cd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:02.502309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92e244-056c-41be-a0c7-c6225a1d56cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:02.547642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6c92e244-056c-41be-a0c7-c6225a1d56cd None None] MFA Rules not processed for user `4337750a9fb940a387400d1f0ca9ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:22:02.548904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6c92e244-056c-41be-a0c7-c6225a1d56cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:22:02.568965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1222/2448] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:22:02.591824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c49262-3f68-4d54-b6bd-2424456fd3d6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:22:02.593776 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38c49262-3f68-4d54-b6bd-2424456fd3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:22:02.608238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c49262-3f68-4d54-b6bd-2424456fd3d6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:22:02.612416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370a43-f46f-4c65-a773-4f33b5b647ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:22:02.613214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370a43-f46f-4c65-a773-4f33b5b647ab service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:02.616229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370a43-f46f-4c65-a773-4f33b5b647ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:02.616229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370a43-f46f-4c65-a773-4f33b5b647ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:02.622296 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42370a43-f46f-4c65-a773-4f33b5b647ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:22:02.623999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42370a43-f46f-4c65-a773-4f33b5b647ab service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:22:02.632094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1227/2449] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:22:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:22:02.721182 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48431e77-a3d0-4e91-afb6-adc6109138ac None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:02.721565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48431e77-a3d0-4e91-afb6-adc6109138ac None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:02.721927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48431e77-a3d0-4e91-afb6-adc6109138ac None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:02.723386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1223/2450] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 10 23:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 23:22:02.729228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af3389-a02e-4f47-bc4f-0f43e8b3a62a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:02.735924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af3389-a02e-4f47-bc4f-0f43e8b3a62a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:02.735924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af3389-a02e-4f47-bc4f-0f43e8b3a62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:02.761662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a5af3389-a02e-4f47-bc4f-0f43e8b3a62a None None] MFA Rules not processed for user `07cef180b05347ebbcc6f3a6e97dac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:22:02.762917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a5af3389-a02e-4f47-bc4f-0f43e8b3a62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:22:02.790212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1228/2451] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 10 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:22:02.805815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48431e77-a3d0-4e91-afb6-adc6109138ac None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:22:02.819668 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48431e77-a3d0-4e91-afb6-adc6109138ac None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:22:02.824721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:22:02.825321 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:02.825753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:02.825878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:02.843896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:22:02.845621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:22:02.857596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1224/2452] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:22:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:22:04.393176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42af4-6465-4a41-b72d-36a176bbc023 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:04.393693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42af4-6465-4a41-b72d-36a176bbc023 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:04.393821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42af4-6465-4a41-b72d-36a176bbc023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:04.413705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-34e42af4-6465-4a41-b72d-36a176bbc023 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:22:04.414929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-34e42af4-6465-4a41-b72d-36a176bbc023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:22:04.422072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1229/2453] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:22:04.438158 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:22:04.450700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36bd9b-f96b-4de9-a716-b8bfd00f056c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:22:04.455584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:22:04.456163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:04.456434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:04.456698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:04.467204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:22:04.468856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:22:04.477814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1225/2454] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:22:14.410168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=21 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:22:14.474333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=28 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:22:25.219844 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:25.220315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:25.220410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:25.239594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:22:25.240938 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:22:25.249033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1230/2455] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 10 23:22:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:22:25.261099 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:22:25.276667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718687ff-7d76-4aae-b30b-c0e791f85f31 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:22:25.282393 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:22:25.283008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:22:25.283277 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:22:25.283539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:22:25.293815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:22:25.295458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:22:25.303415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1226/2456] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:22:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:22:35.234117 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:22:35.300159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:27.303666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:27.313897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286fce8-605d-423c-a962-03781bae6a5f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:27.317578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:27.318178 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:27.318456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:27.318758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:27.320520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:27.354600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:27.356054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:27.376012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1231/2457] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:30.972719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:30.991753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfa9ed-ba76-4bf0-9679-f15a14ab8bf1 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:30.999853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:31.000726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:31.001228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:31.001784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:31.004392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:31.022688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:31.025487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:31.037346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1227/2458] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:31.108999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:31.110489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:31.123524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d3320-efa0-4a70-90e6-ce510230bc99 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:31.127409 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:31.128033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:31.128303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:31.128571 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] PATH_INFO: `/v3/users/22e92975af1d439f88e33cc494c88b1f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:31.143878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e92975af1d439f88e33cc494c88b1f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:31.145795 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:31.309648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e92975af1d439f88e33cc494c88b1f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.321976 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e92975af1d439f88e33cc494c88b1f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.329181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e92975af1d439f88e33cc494c88b1f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.329851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e92975af1d439f88e33cc494c88b1f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.330494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9687ecd4-4fac-4ad6-b186-d19fb43e7b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e92975af1d439f88e33cc494c88b1f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.430154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1232/2459] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:23:31 2022] DELETE /identity/v3/users/22e92975af1d439f88e33cc494c88b1f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:31.682472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:31.695921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db0981-f2b7-4777-a29c-de7274092d7c service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:31.754686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:31.755370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:31.755651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:31.755927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] PATH_INFO: `/v3/projects/6da31a6c1eb84fa1aaf256037c8848e0` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:31.768042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da31a6c1eb84fa1aaf256037c8848e0)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:31.770103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:31.895093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da31a6c1eb84fa1aaf256037c8848e0', 'request_id': 'req-813f2187-1cec-438d-bd31-d9b981d1f407'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.972700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da31a6c1eb84fa1aaf256037c8848e0', 'request_id': 'req-813f2187-1cec-438d-bd31-d9b981d1f407'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.978762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da31a6c1eb84fa1aaf256037c8848e0', 'request_id': 'req-813f2187-1cec-438d-bd31-d9b981d1f407'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.991400 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] The token cache is being invalidate because project 6da31a6c1eb84fa1aaf256037c8848e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:23:31.991682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.992324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-813f2187-1cec-438d-bd31-d9b981d1f407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:31.993775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1228/2460] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:23:31 2022] DELETE /identity/v3/projects/6da31a6c1eb84fa1aaf256037c8848e0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:31.999479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:31.999764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.000033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.060809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:32.062092 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:32.116142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:32.128423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1233/2461] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:32.136206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.136535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.138466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.157465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:32.158675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:32.166537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1229/2462] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:32.171783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.182379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c839fc-4685-4dc6-aba1-3fa569e648c8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.187095 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.187666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.187942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.188212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.188848 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:23:32.189145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.190747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.212647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1234/2463] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:23:32.212777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.216749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225eaf2f-c0c0-4777-8ea3-ebf6eddb3caa None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.222306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.222993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.223216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.223481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.224124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.226068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.256897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.258593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:32.317501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6275037-d2aa-484f-a5f4-1158f029989d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.321897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.322505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.322897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.323171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.324952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:32.357191 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1230/2464] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:32 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:32.362054 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.395949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:32.399050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.400528 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.414316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1235/2465] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:23:32.441864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ba11f-7a1b-4d83-b9b4-d1eb04472aae admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.446738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.447353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.449497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.449497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.449497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.450499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.539035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1231/2466] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:32 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:32.545675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.556219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02a031-d435-46fe-bb34-02581d374110 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.560966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.561634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.561912 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.562181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.562847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.564431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.574155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1236/2467] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:23:32.578617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.588692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42488f10-df3c-4985-b6ec-551d21fc0821 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.593825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496dd47-81d2-4e84-ab2e-5b73f0d18231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.594412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496dd47-81d2-4e84-ab2e-5b73f0d18231 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.594724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496dd47-81d2-4e84-ab2e-5b73f0d18231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.595023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496dd47-81d2-4e84-ab2e-5b73f0d18231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.595659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496dd47-81d2-4e84-ab2e-5b73f0d18231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.597232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496dd47-81d2-4e84-ab2e-5b73f0d18231 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.607022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1232/2468] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:32.611071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.620996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ca2a-bf70-4191-a20e-242f646b502e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.625709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.626349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.626753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.627033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] PATH_INFO: `/v3/projects/bc360e0e01af4b09ba54cff550a19b9e/users/bbba935a966b407b964723983eac5c22/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.642086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc360e0e01af4b09ba54cff550a19b9e, user_id=bbba935a966b407b964723983eac5c22, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.644785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.713627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1237/2469] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:32 2022] PUT /identity/v3/projects/bc360e0e01af4b09ba54cff550a19b9e/users/bbba935a966b407b964723983eac5c22/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:32.724487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.724774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.725048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.752353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] MFA Rules not processed for user `bbba935a966b407b964723983eac5c22`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:32.753478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:32.816584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1233/2470] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:32.828590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:32.830384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:32.885188 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:32.908490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeaea5-a85a-478c-bc52-4601f4eb28bd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:32.911867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:32.912460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:32.912736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:32.913008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:32.921642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:32.923102 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:32.932277 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1238/2471] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:33.601760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:33.663995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cfd6e-5f27-47ab-aae3-524852c9dc7e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:33.667999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:33.671882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:33.671882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:33.671882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:33.671882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:33.722553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:33.743474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:33.744907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:33.755937 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1234/2472] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:39.861174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:39.862701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:39.874642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1247df2-5957-4b35-be2c-44722c4ac6d9 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:39.878285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:39.878909 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:39.879189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:39.879459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:39.881158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:39.944789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:39.946253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf9791-fc52-44fe-96ad-245d75f5b0a5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:39.956364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1239/2473] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:43.247764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:43.265230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e06ca-9612-4691-a0ca-2c4b2357e0d6 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:43.275900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:43.276544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:43.276822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:43.277090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] PATH_INFO: `/v3/users/bbba935a966b407b964723983eac5c22` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:43.278212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbba935a966b407b964723983eac5c22)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:43.280264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:43.526599 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbba935a966b407b964723983eac5c22'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:43.538950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbba935a966b407b964723983eac5c22'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:43.547726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbba935a966b407b964723983eac5c22'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:43.548557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbba935a966b407b964723983eac5c22'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:43.549240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbba935a966b407b964723983eac5c22'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:43.658024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1235/2474] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:23:43 2022] DELETE /identity/v3/users/bbba935a966b407b964723983eac5c22 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:43.662752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:43.663238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:43.663335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:43.682782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:43.685384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:43.699924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:43.701476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:43.767358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1240/2475] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:43.776497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c050b-cf04-4f10-bafb-241a9a2ac91f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:43.777038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:43.781899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:43.782022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:43.782022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:43.782022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:43.782319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] PATH_INFO: `/v3/users/45dd5b35a039468d99ec0d393056c5af` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:43.796125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45dd5b35a039468d99ec0d393056c5af)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:43.798108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:43.837910 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4afc-06bd-4e97-9b3c-70ae39237296 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:43.841469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:43.842070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:43.842346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:43.842617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:43.851503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:43.852917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:43.863000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1241/2476] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:23:44.068223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:44.071564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45dd5b35a039468d99ec0d393056c5af'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.091575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45dd5b35a039468d99ec0d393056c5af'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.105488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45dd5b35a039468d99ec0d393056c5af'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.106100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45dd5b35a039468d99ec0d393056c5af'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.106918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45dd5b35a039468d99ec0d393056c5af'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.228139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1236/2477] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:23:43 2022] DELETE /identity/v3/users/45dd5b35a039468d99ec0d393056c5af => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:44.363323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:44.364919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:44.381706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed05692-511b-4a10-8e8d-cfa51885a60d service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:44.400477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:44.452129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:44.452768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.453152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.453421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] PATH_INFO: `/v3/projects/bc360e0e01af4b09ba54cff550a19b9e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.454582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc360e0e01af4b09ba54cff550a19b9e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:44.456562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:44.473188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:44.475244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:44.496791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4b6a9-9b0d-45f8-9c62-9aafbcaeb5a2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:44.501814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:44.502629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.503052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.503534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] PATH_INFO: `/v3/projects/830b226057d740ca85bbdde6c1a5dca8` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.515950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830b226057d740ca85bbdde6c1a5dca8)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:44.517899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:44.550694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc360e0e01af4b09ba54cff550a19b9e', 'request_id': 'req-895dc58c-97dd-43a0-b4f7-30800c4a9305'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.618059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830b226057d740ca85bbdde6c1a5dca8', 'request_id': 'req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.629195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc360e0e01af4b09ba54cff550a19b9e', 'request_id': 'req-895dc58c-97dd-43a0-b4f7-30800c4a9305'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.635484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc360e0e01af4b09ba54cff550a19b9e', 'request_id': 'req-895dc58c-97dd-43a0-b4f7-30800c4a9305'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.646993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] The token cache is being invalidate because project bc360e0e01af4b09ba54cff550a19b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:23:44.647281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.647926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-895dc58c-97dd-43a0-b4f7-30800c4a9305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.649419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1242/2478] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:23:44 2022] DELETE /identity/v3/projects/bc360e0e01af4b09ba54cff550a19b9e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:44.654041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173501e4-2fdc-4172-8a23-835fa7017d2c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.654324 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173501e4-2fdc-4172-8a23-835fa7017d2c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.655182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173501e4-2fdc-4172-8a23-835fa7017d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.674040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-173501e4-2fdc-4172-8a23-835fa7017d2c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:44.675439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-173501e4-2fdc-4172-8a23-835fa7017d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:44.726606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830b226057d740ca85bbdde6c1a5dca8', 'request_id': 'req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.733116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830b226057d740ca85bbdde6c1a5dca8', 'request_id': 'req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.733671 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1243/2479] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:44.742894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.743182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.743456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.745380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] The token cache is being invalidate because project 830b226057d740ca85bbdde6c1a5dca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:23:44.745670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.746355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ac42f6a3-7ac3-43a4-9e9e-e4ea198ae698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:44.747906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1237/2480] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:23:44 2022] DELETE /identity/v3/projects/830b226057d740ca85bbdde6c1a5dca8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:23:44.753710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ca842-00da-4380-aed0-df9c18fa655d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.753987 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ca842-00da-4380-aed0-df9c18fa655d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.754258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ca842-00da-4380-aed0-df9c18fa655d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.763695 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:44.765506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:44.773497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1244/2481] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:44.773875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c77ca842-00da-4380-aed0-df9c18fa655d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:44.775068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c77ca842-00da-4380-aed0-df9c18fa655d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:44.777510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:44.783626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1238/2482] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:44.789501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da3e34-e25b-441e-a682-f1707d3996c2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:44.791254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.791546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.791821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.795159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:44.795761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.796040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.796308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.796948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:23:44.797248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:44.799231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:44.811951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1245/2483] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:23:44.813466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:44.814921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:44.817205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:44.830282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1239/2484] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:44.830535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442615a-9c3a-416c-9431-96ec55c1201a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:44.834251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:44.835764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:44.836700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.836983 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.837259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.837882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:44.841496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:44.844720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81384569-27a0-4225-9f14-d7e7d47454f1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:44.849871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:44.850745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.850980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.851253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.851906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:23:44.852362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:44.855036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:44.867555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1240/2485] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:23:44.871970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:44.883058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d0a4-de62-49ab-80d7-571760288a78 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:44.887770 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:44.888355 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:44.888651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:44.889454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:44.890098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:44.892078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:44.933632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:44.940510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1246/2486] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:44.945036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.006221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffdb9a-266a-4ccf-ba42-3419e360243b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.011701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.012304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.012580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.012845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.013456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.014891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1241/2487] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:23:45.016011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.019278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.066351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:45.080429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b403c-4679-420f-860a-8fa7b45c2a86 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.085086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.085685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.085964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.086227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.087133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.089055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.104200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1247/2488] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:44 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:23:45.109084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.118623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f8b73-ba10-44df-b88b-5e463de4e21f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.123232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.123843 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.124122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.124396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.125036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.126620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.136241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1248/2489] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:45.140322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.149400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5c6b2-06eb-4e87-b2b9-750da1fdee3e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.153894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.155034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.155319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.155655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.156273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.157819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.168143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1249/2490] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:45.173594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.184253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2a87c-6b1d-4eb6-b331-43ab090c4915 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.189919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f55d90-b915-4b21-8ea2-201bdafae054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.193705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55d90-b915-4b21-8ea2-201bdafae054 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.193705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55d90-b915-4b21-8ea2-201bdafae054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.193705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55d90-b915-4b21-8ea2-201bdafae054 admin admin] PATH_INFO: `/v3/projects/4536fd3e8f21403dabc072115fa8d3a9/users/bb5eca5f2b42440a984e49bd7ba1b7ba/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.199163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1242/2491] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:45 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:45.200512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.207919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f55d90-b915-4b21-8ea2-201bdafae054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4536fd3e8f21403dabc072115fa8d3a9, user_id=bb5eca5f2b42440a984e49bd7ba1b7ba, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.210565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f55d90-b915-4b21-8ea2-201bdafae054 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.210754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cdec9-24bc-42df-9fd3-e917ca66d31d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.215659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.216247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.216528 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.216800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.218100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.219711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.229667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1243/2492] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:45.233684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.243183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14db95-0802-4b53-81c0-80a49d9e6687 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.248040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.248640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.248917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.249188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.249806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.251462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.261652 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1244/2493] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:45.266082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.275213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89870-7154-4f88-9b40-725f845b5376 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.280775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.281010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.281282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.281547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] PATH_INFO: `/v3/projects/f3e1c4732e4943cc85c7ce3a774815b3/users/4f8d6aeeab0042e48e86fc79e4a3ba56/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.296604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e1c4732e4943cc85c7ce3a774815b3, user_id=4f8d6aeeab0042e48e86fc79e4a3ba56, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.299424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.321284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1250/2494] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:45 2022] PUT /identity/v3/projects/4536fd3e8f21403dabc072115fa8d3a9/users/bb5eca5f2b42440a984e49bd7ba1b7ba/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:23:45.332137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44587bda-7533-40db-8f26-6622368fee5f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.332421 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44587bda-7533-40db-8f26-6622368fee5f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.332696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44587bda-7533-40db-8f26-6622368fee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.360906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-44587bda-7533-40db-8f26-6622368fee5f None None] MFA Rules not processed for user `bb5eca5f2b42440a984e49bd7ba1b7ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:45.362130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44587bda-7533-40db-8f26-6622368fee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:45.424231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1245/2495] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:45 2022] PUT /identity/v3/projects/f3e1c4732e4943cc85c7ce3a774815b3/users/4f8d6aeeab0042e48e86fc79e4a3ba56/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:23:45.425236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1251/2496] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:45.429572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.434145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.434434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.434770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.464973 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] MFA Rules not processed for user `4f8d6aeeab0042e48e86fc79e4a3ba56`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:45.466284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:45.486339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:45.500064 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabcd8f-8f2f-4915-a964-424d88753e99 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.505113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.505693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.505972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.506237 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.506954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.508839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.535496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1252/2497] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:45.546758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.548177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:45.602714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1246/2498] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:45 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:23:45.607170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.622904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4886c-0c7f-4451-a93d-4143eddf77bc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.670817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b289a3c-c73e-4619-8327-29da9583b111 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.675507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.676101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.676378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.676655 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.677376 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.679329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.689428 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.690044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.690318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.690590 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.745254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.746847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.750273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:45.756171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1253/2499] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:23:45.764220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1247/2500] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:45 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:45.765387 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.780535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09582f78-a01a-493e-9703-62c88ace32c2 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.787356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.788322 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.788713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.788981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.790016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.791281 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.810865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1254/2501] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:23:45.820248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.829935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:45.835252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ac825-2e85-4994-abb1-9cd520b807a4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.840830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:45.842333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:45.842785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:45.843126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] PATH_INFO: `/v3/projects/be91a1b1d47540aca62fb5ace241a5da/users/d5eab78edba642a0806f833e15928bd7/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:45.866649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be91a1b1d47540aca62fb5ace241a5da, user_id=d5eab78edba642a0806f833e15928bd7, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:45.870151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:45.982512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:45.992743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48cca-8b8f-4a6c-bb43-28147af1f9a0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:45.997355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:46.001457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.001457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.001457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] PATH_INFO: `/v3/users/4f8d6aeeab0042e48e86fc79e4a3ba56` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.001457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8d6aeeab0042e48e86fc79e4a3ba56)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:46.001922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:46.030348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1248/2502] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:45 2022] PUT /identity/v3/projects/be91a1b1d47540aca62fb5ace241a5da/users/d5eab78edba642a0806f833e15928bd7/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:46.034297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:46.098286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074d704-d8ce-4b59-a0b1-3eb0b481da75 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:46.103118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:46.103772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.103994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.104262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.104928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:46.106517 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:46.116958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1249/2503] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:46.121155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:46.132046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae301b4-f43f-4a3b-a2db-7e4c13f3788d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:46.136881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dcb71-0278-4f75-a786-c7a0b7adefc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:46.141168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dcb71-0278-4f75-a786-c7a0b7adefc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.141168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dcb71-0278-4f75-a786-c7a0b7adefc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.141168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dcb71-0278-4f75-a786-c7a0b7adefc7 admin admin] PATH_INFO: `/v3/projects/be91a1b1d47540aca62fb5ace241a5da/users/d5eab78edba642a0806f833e15928bd7/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.141168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dcb71-0278-4f75-a786-c7a0b7adefc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be91a1b1d47540aca62fb5ace241a5da, user_id=d5eab78edba642a0806f833e15928bd7, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:46.143210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dcb71-0278-4f75-a786-c7a0b7adefc7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:46.153653 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1250/2504] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:46 2022] PUT /identity/v3/projects/be91a1b1d47540aca62fb5ace241a5da/users/d5eab78edba642a0806f833e15928bd7/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:46.163903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.164185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.164458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.192543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] MFA Rules not processed for user `d5eab78edba642a0806f833e15928bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:46.198553 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.240772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8d6aeeab0042e48e86fc79e4a3ba56'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:46.254437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8d6aeeab0042e48e86fc79e4a3ba56'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:46.262343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8d6aeeab0042e48e86fc79e4a3ba56'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:46.263260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8d6aeeab0042e48e86fc79e4a3ba56'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:46.263984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25324fc4-b78b-442b-ac84-67a6ace4fecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8d6aeeab0042e48e86fc79e4a3ba56'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:46.275951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1251/2505] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:46.287585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:46.288950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.344828 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1255/2506] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:23:45 2022] DELETE /identity/v3/users/4f8d6aeeab0042e48e86fc79e4a3ba56 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:23:46.349314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.349672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.350055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.351373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:46.355512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee4d1b-470f-4a52-9fc2-f128cf3e8162 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:46.371130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:46.372437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.409183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:46.409788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.410063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.410328 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.412323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.449416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1256/2507] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:23:46.460177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:46.461979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.483827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:46.485481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:46.496346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1252/2508] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:23:46.525253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2e55-5a5f-491c-9a6e-70cb97f57cab None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:46.529009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:46.530159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.530438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.530750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.546381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:46.548857 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:46.567969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1257/2509] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:23:46.753065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:46.754456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.766429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd844a-2c86-41d6-a9d7-01c7974dfd18 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:46.770320 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:46.771059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:46.771340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:46.771629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:46.774272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:46.838932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:46.840395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b07b5-221f-481f-a331-1ae7c399eb74 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:46.850337 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1253/2510] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:47.004364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.006471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:47.019763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:47.026264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1249b-cc4e-47b6-8155-33205e404a46 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.032558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.032558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.032558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.032558 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] PATH_INFO: `/v3/projects/f3e1c4732e4943cc85c7ce3a774815b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.034153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3e1c4732e4943cc85c7ce3a774815b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.035550 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:47.161254 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3e1c4732e4943cc85c7ce3a774815b3', 'request_id': 'req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:47.264838 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3e1c4732e4943cc85c7ce3a774815b3', 'request_id': 'req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:47.271929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3e1c4732e4943cc85c7ce3a774815b3', 'request_id': 'req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:47.286618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] The token cache is being invalidate because project f3e1c4732e4943cc85c7ce3a774815b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:23:47.286904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:47.287559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da5cfe98-55b7-46ce-9a45-0c4ab0292a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:23:47.289429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1258/2511] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:23:47 2022] DELETE /identity/v3/projects/f3e1c4732e4943cc85c7ce3a774815b3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:47.294126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.294427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.294731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.329209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:47.330434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:47.381430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:47.393485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1254/2512] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:47.402802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.403084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.403354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.425860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:47.427021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:47.443486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1259/2513] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:47.443610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.450881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3f04-4998-46b9-9fb2-041f3c5a8f7d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.455495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.456082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.456362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.456637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.457294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:23:47.457575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.459264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:47.470698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1255/2514] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:23:47.475037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.484864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce0040-1758-4690-b33d-95b43279b25e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.490054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.490640 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.490982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.491253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.491870 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.493747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:47.602546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1260/2515] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:47 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:23:47.609500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.619981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323a65c-691b-4f9c-bb01-6dc0d4902b8c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.675854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.676494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.676714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.676977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.677592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.679586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:47.789699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1256/2516] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:47 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:47.793852 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.804947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006799fb-16eb-4a4c-bd4c-aaf1ee9f03b6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.809700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.810321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.810679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.815796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.815796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.815796 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:47.823853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1261/2517] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:47.827841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.838033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca4375-48c7-45b9-bafd-e7b9921fd47f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.843145 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b972ac-124c-4e83-9bf7-78fe91202968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.843723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b972ac-124c-4e83-9bf7-78fe91202968 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.843996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b972ac-124c-4e83-9bf7-78fe91202968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.844262 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b972ac-124c-4e83-9bf7-78fe91202968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.844996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b972ac-124c-4e83-9bf7-78fe91202968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.846637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b972ac-124c-4e83-9bf7-78fe91202968 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:47.856920 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1257/2518] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:23:47.861740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:47.872425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4318766-002b-473d-92c0-661d46dee084 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:47.877219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:47.877871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:47.878151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:47.878426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] PATH_INFO: `/v3/projects/7567940d639c4683ae2a6a3260dfdd25/users/0ff2e760636344eb94ceb442137382b8/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:47.892720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7567940d639c4683ae2a6a3260dfdd25, user_id=0ff2e760636344eb94ceb442137382b8, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:47.895398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:48.010594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1262/2519] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:47 2022] PUT /identity/v3/projects/7567940d639c4683ae2a6a3260dfdd25/users/0ff2e760636344eb94ceb442137382b8/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:48.020888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.021178 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.021526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.050975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] MFA Rules not processed for user `0ff2e760636344eb94ceb442137382b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:48.052224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:48.118042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1258/2520] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:48.123030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:48.187985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f82f1-d912-4b6f-8ae7-cda6d4b95c9e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:48.192696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:48.193290 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.193565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.193834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.194455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:48.196390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:48.320817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1263/2521] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:48 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:48.325205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:48.339068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847e4f0-1fbe-4d9d-8398-b203aa8dab4a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:48.387493 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:48.403339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:48.403946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.404227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.404499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.405279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:48.407269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:48.510891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1259/2522] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:48 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:48.512249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:48.524199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fcd5c-3f44-4d7b-b180-60988e016903 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:48.528877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:48.529981 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.530263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.530537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.531219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:48.532802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:48.534138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:48.543922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1264/2523] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:23:48.548494 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:48.558814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7650-bffa-4e8e-ad93-caa0e6664906 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:48.564052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e05da-195e-4d1f-a1a5-a37c8ee8b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:48.564680 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e05da-195e-4d1f-a1a5-a37c8ee8b522 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.564922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e05da-195e-4d1f-a1a5-a37c8ee8b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.565193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e05da-195e-4d1f-a1a5-a37c8ee8b522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.565834 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e05da-195e-4d1f-a1a5-a37c8ee8b522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:48.567520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e05da-195e-4d1f-a1a5-a37c8ee8b522 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:48.578287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1260/2524] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:23:48.582534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:48.592840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f1e2-6b46-45fd-a1de-28641d02e29f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:48.597435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:48.598071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.598346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.598610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] PATH_INFO: `/v3/projects/f14665159d914caa8f55299546511ea1/users/fd1da3fb62544ab7873b21f2e1031be2/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.614730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14665159d914caa8f55299546511ea1, user_id=fd1da3fb62544ab7873b21f2e1031be2, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:48.617679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:48.700525 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1265/2525] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:48 2022] PUT /identity/v3/projects/f14665159d914caa8f55299546511ea1/users/fd1da3fb62544ab7873b21f2e1031be2/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:48.711918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.712362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.712779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.741354 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] MFA Rules not processed for user `fd1da3fb62544ab7873b21f2e1031be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:48.742710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:48.807482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1261/2526] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:48.813173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:48.874472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4f946-d0bc-443c-8738-7f4a4f83b9d0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:48.880772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:48.883138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:48.883423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:48.883696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:48.884318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:48.887880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.013013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1266/2527] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:23:48 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:49.016970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:49.027847 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad107c-12bb-4bcb-b0d9-9dd19a1a12f3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:49.140899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:49.142164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.149424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.149424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.149424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:49.149424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.275434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1262/2528] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:23:49 2022] POST /identity/v3/users => generated 300 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:23:49.280022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:49.290479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3575-33f9-4e70-85a5-02ae53698230 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:49.295709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:49.296511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.296809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.297067 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.297706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:49.299339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.310684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1267/2529] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:23:49.314792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:49.325149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116fa40-012c-480b-819b-9d9c700ddbc6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:49.330060 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:49.330732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.331017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.331305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] PATH_INFO: `/v3/projects/1c45bf49089848b397e1460c855b0e38/users/e7ebf0527d4047a4be3c20acb3c3acf4/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.346986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c45bf49089848b397e1460c855b0e38, user_id=e7ebf0527d4047a4be3c20acb3c3acf4, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:49.349672 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.447272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1263/2530] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:49 2022] PUT /identity/v3/projects/1c45bf49089848b397e1460c855b0e38/users/e7ebf0527d4047a4be3c20acb3c3acf4/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:49.451434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:49.461792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58627098-a858-4363-96cb-8fbb6f60ae72 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:49.518079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fbd37-df7d-4f2c-83d1-49efc0cf7144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:49.518731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fbd37-df7d-4f2c-83d1-49efc0cf7144 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.519032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fbd37-df7d-4f2c-83d1-49efc0cf7144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.519368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fbd37-df7d-4f2c-83d1-49efc0cf7144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.519924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fbd37-df7d-4f2c-83d1-49efc0cf7144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:49.521511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fbd37-df7d-4f2c-83d1-49efc0cf7144 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.532369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1268/2531] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:23:49.536713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:49.547895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ce64a-a496-4a8c-abf7-5a5083958a2c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:49.552538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:49.553179 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.553456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.556677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] PATH_INFO: `/v3/projects/1c45bf49089848b397e1460c855b0e38/users/e7ebf0527d4047a4be3c20acb3c3acf4/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.556677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c45bf49089848b397e1460c855b0e38, user_id=e7ebf0527d4047a4be3c20acb3c3acf4, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:49.558310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.568994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1264/2532] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:23:49 2022] PUT /identity/v3/projects/1c45bf49089848b397e1460c855b0e38/users/e7ebf0527d4047a4be3c20acb3c3acf4/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:23:49.581248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.581663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.582000 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.610411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] MFA Rules not processed for user `e7ebf0527d4047a4be3c20acb3c3acf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:23:49.611762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:49.643825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:49.681312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1269/2533] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:23:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:23:49.694137 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:49.695848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:49.707102 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:23:49.752597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf3f81-9f62-4987-a94e-f4f327682da0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:49.757070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:49.757691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:49.757972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:49.758275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:49.767676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:49.769103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:49.778285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1265/2534] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:51.263815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:51.273742 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fc221-d678-4b1a-8f2b-2b93dc992871 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:51.277425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:51.280932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:51.280932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:51.280932 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:51.334456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:51.335882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:51.344884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1270/2535] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:23:59.461919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:23:59.463332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:23:59.474514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66ec66-a5c9-4ad2-8ee7-7ed90c01ef77 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:23:59.478084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:23:59.478685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:23:59.478972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:23:59.479247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:23:59.536922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:23:59.538371 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:23:59.548155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1266/2536] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:23:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:00.262495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:00.263944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:00.332358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6083-ce84-49d8-aeb4-1cc5621e91ae service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:00.336340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:00.336947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:00.337223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:00.337492 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:00.339256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:00.349537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:00.350998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb8a0-4d81-452b-8816-dd1482e07cc0 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:00.359865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1271/2537] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:01.250903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:01.252941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:01.349252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278f154-c2ac-4930-853a-41dda53c4446 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:01.355589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:01.356635 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:01.357794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:01.358907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] PATH_INFO: `/v3/users/bb5eca5f2b42440a984e49bd7ba1b7ba` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:01.360784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5eca5f2b42440a984e49bd7ba1b7ba)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:01.363049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:01.646297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5eca5f2b42440a984e49bd7ba1b7ba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:01.650745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:01.660373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5eca5f2b42440a984e49bd7ba1b7ba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:01.667299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5eca5f2b42440a984e49bd7ba1b7ba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:01.667990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5eca5f2b42440a984e49bd7ba1b7ba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:01.668598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5eca5f2b42440a984e49bd7ba1b7ba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:01.773246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1267/2538] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:01 2022] DELETE /identity/v3/users/bb5eca5f2b42440a984e49bd7ba1b7ba => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:01.782581 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:01.783273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:01.783273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:01.801821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:01.803072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:01.861064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1272/2539] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:01.871362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:01.872882 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:01.931206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bd9f9-6aa6-4133-8d77-eef79f173693 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:01.935025 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:01.935979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:01.936255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:01.936584 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:01.946373 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:01.947880 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:01.958563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1268/2540] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:02.368309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:02.369766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:02.383929 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239ab6-fae4-4ffd-ad25-df2f18789a73 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:02.388621 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:02.390380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:02.390864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:02.391145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] PATH_INFO: `/v3/projects/4536fd3e8f21403dabc072115fa8d3a9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:02.392378 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4536fd3e8f21403dabc072115fa8d3a9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:02.394925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:02.397867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:02.516394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4536fd3e8f21403dabc072115fa8d3a9', 'request_id': 'req-da0a0469-6a92-494c-9344-317c1424771f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.554735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4536fd3e8f21403dabc072115fa8d3a9', 'request_id': 'req-da0a0469-6a92-494c-9344-317c1424771f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.561031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4536fd3e8f21403dabc072115fa8d3a9', 'request_id': 'req-da0a0469-6a92-494c-9344-317c1424771f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.571858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] The token cache is being invalidate because project 4536fd3e8f21403dabc072115fa8d3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:02.572146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.572771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.574257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1273/2541] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:02 2022] DELETE /identity/v3/projects/4536fd3e8f21403dabc072115fa8d3a9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:02.578756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:02.588764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cad731-7463-49a3-bd29-152933164029 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:02.646089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:02.646899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:02.647176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:02.647565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] PATH_INFO: `/v3/users/d5eab78edba642a0806f833e15928bd7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:02.648746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5eab78edba642a0806f833e15928bd7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:02.650701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:02.832340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5eab78edba642a0806f833e15928bd7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.844980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5eab78edba642a0806f833e15928bd7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.852253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5eab78edba642a0806f833e15928bd7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.852974 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5eab78edba642a0806f833e15928bd7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.853572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2e453d9a-83a2-4f1b-ac11-9261240d8deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5eab78edba642a0806f833e15928bd7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:02.855410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:03.006503 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1269/2542] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:02 2022] DELETE /identity/v3/users/d5eab78edba642a0806f833e15928bd7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:03.481987 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:03.484025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:03.498565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0469-6a92-494c-9344-317c1424771f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:03.556425 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:03.557097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:03.557373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:03.557639 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] PATH_INFO: `/v3/projects/be91a1b1d47540aca62fb5ace241a5da` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:03.558766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be91a1b1d47540aca62fb5ace241a5da)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:03.560668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:03.675855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be91a1b1d47540aca62fb5ace241a5da', 'request_id': 'req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:03.766846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be91a1b1d47540aca62fb5ace241a5da', 'request_id': 'req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:03.772907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be91a1b1d47540aca62fb5ace241a5da', 'request_id': 'req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:03.784412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] The token cache is being invalidate because project be91a1b1d47540aca62fb5ace241a5da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:03.784708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:03.785373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5104647f-7a6f-4f1b-b7f9-b3cdb9d51ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:03.786927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1274/2543] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:03 2022] DELETE /identity/v3/projects/be91a1b1d47540aca62fb5ace241a5da => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:03.791905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:03.792388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:03.792489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:03.812564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:03.816794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:03.873072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1270/2544] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:03.881668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:03.881956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:03.882234 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:03.901207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:03.902861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:03.910591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1275/2545] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:03.914980 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:03.925207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f93f2-921e-49cf-9eab-e55e5be7553e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:03.930557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:03.931188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:03.931570 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:03.931977 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:03.932620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:03.932915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:03.934489 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:03.946890 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1271/2546] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:24:03.951137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:03.960365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633adf21-f25c-4d7d-9d39-6c0726dc47e4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:03.965379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:03.965963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:03.966242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:03.966515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:03.967184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:03.969086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:03.984802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:04.105192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1276/2547] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:03 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:04.110003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:04.120091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc196c-83c0-4f86-b07f-feccd52a71d7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:04.176648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:04.177242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.177520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.177789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.178406 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:04.180347 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:04.275010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1272/2548] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:04 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:04.279285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:04.289943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717cd44-7ebc-4b1c-8df1-9b986b557ed8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:04.294572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:04.301032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.301032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.301032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.301032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:04.301032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:04.308785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1277/2549] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:04.313098 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:04.323492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4d9dd-733f-46ef-a1f3-240db654181e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:04.328150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:04.328731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.329004 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.329271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.329901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:04.331758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:04.343126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1273/2550] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:04.351110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:04.362536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aa27e-93c5-4d71-b898-8a962e87e0f2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:04.367364 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcb766-0b9d-4759-a011-59efac7e7ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:04.368008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcb766-0b9d-4759-a011-59efac7e7ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.368280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcb766-0b9d-4759-a011-59efac7e7ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.368546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcb766-0b9d-4759-a011-59efac7e7ea8 admin admin] PATH_INFO: `/v3/projects/24bf7aa6615a4b0aade4dd7dcdd2bbeb/users/a77d601c64b940bab3a213e746970cc4/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.383159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcb766-0b9d-4759-a011-59efac7e7ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24bf7aa6615a4b0aade4dd7dcdd2bbeb, user_id=a77d601c64b940bab3a213e746970cc4, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:04.385865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcb766-0b9d-4759-a011-59efac7e7ea8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:04.453874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:04.455731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:04.469103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452afd7-7ace-46ab-9d49-380696a8a851 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:04.475775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1278/2551] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:04 2022] PUT /identity/v3/projects/24bf7aa6615a4b0aade4dd7dcdd2bbeb/users/a77d601c64b940bab3a213e746970cc4/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:04.495180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.495793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.495793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.522922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:04.523868 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] MFA Rules not processed for user `a77d601c64b940bab3a213e746970cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:04.525232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:04.529021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:04.529724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.529946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.530209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] PATH_INFO: `/v3/users/0ff2e760636344eb94ceb442137382b8` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.531427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff2e760636344eb94ceb442137382b8)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:04.533339 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:04.589379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1279/2552] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:04.593849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:04.604244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3a7a-e2b8-453d-8ed1-272a4afa24e0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:04.609366 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:04.609946 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:04.610224 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:04.610493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:04.611136 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:04.613027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:04.962464 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1280/2553] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:04 2022] POST /identity/v3/projects => generated 389 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:04.966500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.003707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff2e760636344eb94ceb442137382b8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:05.015891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff2e760636344eb94ceb442137382b8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:05.022574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff2e760636344eb94ceb442137382b8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:05.023282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff2e760636344eb94ceb442137382b8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:05.023887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06eacc0d-d2c1-4324-8163-46f57cda6c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff2e760636344eb94ceb442137382b8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:05.031130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bf29a-4338-49f5-bd53-eb468e0befda admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.035915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.039493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.039493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.039493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.039493 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.039858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.097836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1274/2554] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:04 2022] DELETE /identity/v3/users/0ff2e760636344eb94ceb442137382b8 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:05.102069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.102350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.102768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.122458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:05.125055 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:05.127221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1281/2555] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:04 2022] POST /identity/v3/users => generated 301 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:05.132186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.133965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:05.146965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcea02c-5a58-4c35-b84b-947b0106a97e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.159328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:05.188412 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1275/2556] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:05.199207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.200585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:05.204232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.207975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.207975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.207975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.207975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.207975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.222696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1282/2557] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:24:05.222696 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.232545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8344bfa-de10-44bb-a1c7-5f161828ac90 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.237825 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.238482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.238801 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.239077 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] PATH_INFO: `/v3/projects/e95492b7aec54e2081b90d17db81cedc/users/5d3c45df07c143c4a00d9a06f03200d5/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.254539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95492b7aec54e2081b90d17db81cedc, user_id=5d3c45df07c143c4a00d9a06f03200d5, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.257259 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.260424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c23018-fb8b-475c-896d-230996ca13fb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.264162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.264810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.265089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.265361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.274699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.276129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.286274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1276/2558] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:24:05.359210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1283/2559] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:05 2022] PUT /identity/v3/projects/e95492b7aec54e2081b90d17db81cedc/users/5d3c45df07c143c4a00d9a06f03200d5/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:05.363288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.372954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154bc80-e2cb-4c4c-b581-1c6374ee38d1 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.390424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:05.430485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b2d54-5013-499b-867d-912b34bfbd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.431122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b2d54-5013-499b-867d-912b34bfbd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.431402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b2d54-5013-499b-867d-912b34bfbd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.431675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b2d54-5013-499b-867d-912b34bfbd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.432316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b2d54-5013-499b-867d-912b34bfbd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.433928 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b2d54-5013-499b-867d-912b34bfbd89 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.443926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1277/2560] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:05.448231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.459302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3c475-2a88-4523-9fca-33afc8176842 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.464438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.465091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.465372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.465645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] PATH_INFO: `/v3/projects/e95492b7aec54e2081b90d17db81cedc/users/5d3c45df07c143c4a00d9a06f03200d5/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.467791 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95492b7aec54e2081b90d17db81cedc, user_id=5d3c45df07c143c4a00d9a06f03200d5, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.470397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.481002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1284/2561] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:05 2022] PUT /identity/v3/projects/e95492b7aec54e2081b90d17db81cedc/users/5d3c45df07c143c4a00d9a06f03200d5/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:05.493051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.494917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.494917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.524629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] MFA Rules not processed for user `5d3c45df07c143c4a00d9a06f03200d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:05.525850 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:05.594693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1278/2562] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:05.605652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.607433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:05.670731 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d6e37-9d8e-451a-9067-cdc37aa27b48 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.674407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.675056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.675337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.675608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.684406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.685814 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.695623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1285/2563] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:05.749552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.751293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:05.824425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c934a7d-e45c-413b-9fc5-e7bab0ed8877 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.829641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.830387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.830691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.830969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] PATH_INFO: `/v3/projects/7567940d639c4683ae2a6a3260dfdd25` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.832136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7567940d639c4683ae2a6a3260dfdd25)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.834024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.846879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:05.857385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00122fd6-eb43-4394-869d-fa6287536fd2 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:05.862093 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:05.862743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:05.863021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:05.863318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] PATH_INFO: `/v3/users/a77d601c64b940bab3a213e746970cc4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:05.864448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a77d601c64b940bab3a213e746970cc4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:05.866365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:05.945104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:05.948793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7567940d639c4683ae2a6a3260dfdd25', 'request_id': 'req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.033127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7567940d639c4683ae2a6a3260dfdd25', 'request_id': 'req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.039288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7567940d639c4683ae2a6a3260dfdd25', 'request_id': 'req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.051691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] The token cache is being invalidate because project 7567940d639c4683ae2a6a3260dfdd25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:06.051983 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.052625 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.054106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1279/2564] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:05 2022] DELETE /identity/v3/projects/7567940d639c4683ae2a6a3260dfdd25 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:06.058123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:06.058253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a77d601c64b940bab3a213e746970cc4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.059458 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:06.071143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a77d601c64b940bab3a213e746970cc4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.076773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:06.079654 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a77d601c64b940bab3a213e746970cc4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.080384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a77d601c64b940bab3a213e746970cc4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.081014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4c15245d-5315-4c7e-aa93-5175376b2ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a77d601c64b940bab3a213e746970cc4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.122841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acf795-7d6c-4ca7-8100-f97b5c2050e6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:06.128070 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:06.130069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:06.130069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:06.130069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] PATH_INFO: `/v3/users/fd1da3fb62544ab7873b21f2e1031be2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:06.130439 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1da3fb62544ab7873b21f2e1031be2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:06.132253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:06.154279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1286/2565] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:05 2022] DELETE /identity/v3/users/a77d601c64b940bab3a213e746970cc4 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:06.158575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:06.158882 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:06.159169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:06.177920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:06.179389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:06.237012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1287/2566] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:06.246948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:06.248280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:06.304252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888744e6-6acf-443d-a579-2e0b8152f97f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:06.307846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:06.308453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:06.308734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:06.309006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:06.312667 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1da3fb62544ab7873b21f2e1031be2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.318743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:06.320170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:06.324915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1da3fb62544ab7873b21f2e1031be2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.329579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1288/2567] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:24:06.331712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1da3fb62544ab7873b21f2e1031be2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.332448 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1da3fb62544ab7873b21f2e1031be2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.333078 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1da3fb62544ab7873b21f2e1031be2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:06.435155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1280/2568] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:06 2022] DELETE /identity/v3/users/fd1da3fb62544ab7873b21f2e1031be2 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:06.806219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:06.808338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:06.822459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a214e77-00f6-46be-af09-5b0e76fa0a84 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:06.881273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:06.881903 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:06.882185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:06.882458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] PATH_INFO: `/v3/projects/24bf7aa6615a4b0aade4dd7dcdd2bbeb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:06.883609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=24bf7aa6615a4b0aade4dd7dcdd2bbeb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:06.885496 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:06.964364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:06.965895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:06.979912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14486080-9e94-41c0-b322-da998356e05e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:06.984657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:06.985397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:06.985545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:06.985852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] PATH_INFO: `/v3/projects/f14665159d914caa8f55299546511ea1` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:06.987010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14665159d914caa8f55299546511ea1)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:06.989142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:07.007501 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:07.017255 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24bf7aa6615a4b0aade4dd7dcdd2bbeb', 'request_id': 'req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.159660 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:07.166788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14665159d914caa8f55299546511ea1', 'request_id': 'req-080bfb83-6e86-4e43-9449-e6f28a7ef659'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.191315 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24bf7aa6615a4b0aade4dd7dcdd2bbeb', 'request_id': 'req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.193985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24bf7aa6615a4b0aade4dd7dcdd2bbeb', 'request_id': 'req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.205516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] The token cache is being invalidate because project 24bf7aa6615a4b0aade4dd7dcdd2bbeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:07.205804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.206449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.207943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1289/2569] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:06 2022] DELETE /identity/v3/projects/24bf7aa6615a4b0aade4dd7dcdd2bbeb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:07.212104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:07.213413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:07.275756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04c85b-90cf-4d62-84f0-56c6e8eb9abb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:07.330350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:07.331110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:07.331415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:07.331689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] PATH_INFO: `/v3/users/5d3c45df07c143c4a00d9a06f03200d5` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:07.332834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d3c45df07c143c4a00d9a06f03200d5)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:07.334904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:07.343276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14665159d914caa8f55299546511ea1', 'request_id': 'req-080bfb83-6e86-4e43-9449-e6f28a7ef659'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.349423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14665159d914caa8f55299546511ea1', 'request_id': 'req-080bfb83-6e86-4e43-9449-e6f28a7ef659'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.360698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] The token cache is being invalidate because project f14665159d914caa8f55299546511ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:07.360984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.363324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.363324 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1281/2570] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:06 2022] DELETE /identity/v3/projects/f14665159d914caa8f55299546511ea1 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:07.367097 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:07.368476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:07.381931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bfb83-6e86-4e43-9449-e6f28a7ef659 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:07.386477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:07.387123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:07.387441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:07.387715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] PATH_INFO: `/v3/users/e7ebf0527d4047a4be3c20acb3c3acf4` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:07.388824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ebf0527d4047a4be3c20acb3c3acf4)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:07.391457 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:07.614285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d3c45df07c143c4a00d9a06f03200d5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.626849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d3c45df07c143c4a00d9a06f03200d5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.633618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d3c45df07c143c4a00d9a06f03200d5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.634318 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d3c45df07c143c4a00d9a06f03200d5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.634947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d3c45df07c143c4a00d9a06f03200d5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.688903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ebf0527d4047a4be3c20acb3c3acf4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.700775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ebf0527d4047a4be3c20acb3c3acf4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.707569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ebf0527d4047a4be3c20acb3c3acf4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.708248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ebf0527d4047a4be3c20acb3c3acf4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.708874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ebf0527d4047a4be3c20acb3c3acf4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:07.713339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1290/2571] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:07 2022] DELETE /identity/v3/users/5d3c45df07c143c4a00d9a06f03200d5 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:07.799437 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1282/2572] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:07 2022] DELETE /identity/v3/users/e7ebf0527d4047a4be3c20acb3c3acf4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:07.818258 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:08.268001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:08.269645 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:08.336886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a387a4d-4333-473c-925a-3f178496fc2c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:08.482821 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:08.484924 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:08.536746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:08.537381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.537662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.537934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] PATH_INFO: `/v3/projects/e95492b7aec54e2081b90d17db81cedc` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.539127 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95492b7aec54e2081b90d17db81cedc)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:08.541037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:08.558326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d2f0f-9e37-44bd-bfe7-e2c708f24f0d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:08.564161 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:08.564778 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.565088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.565594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] PATH_INFO: `/v3/projects/1c45bf49089848b397e1460c855b0e38` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.566466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c45bf49089848b397e1460c855b0e38)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:08.568394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:08.671556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95492b7aec54e2081b90d17db81cedc', 'request_id': 'req-1608d8b1-aeb3-429e-b739-92335a41a1ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.697434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:08.698720 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c45bf49089848b397e1460c855b0e38', 'request_id': 'req-f95b5999-1abe-45a3-a768-fb459595dd0b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.735940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95492b7aec54e2081b90d17db81cedc', 'request_id': 'req-1608d8b1-aeb3-429e-b739-92335a41a1ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.741835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95492b7aec54e2081b90d17db81cedc', 'request_id': 'req-1608d8b1-aeb3-429e-b739-92335a41a1ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.752608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] The token cache is being invalidate because project e95492b7aec54e2081b90d17db81cedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:08.753991 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.754638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1608d8b1-aeb3-429e-b739-92335a41a1ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.756145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1291/2573] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:08 2022] DELETE /identity/v3/projects/e95492b7aec54e2081b90d17db81cedc => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:08.760771 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f394-35fe-4ae4-8bb2-23bbcd4a92b3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.761076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f394-35fe-4ae4-8bb2-23bbcd4a92b3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.761338 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f394-35fe-4ae4-8bb2-23bbcd4a92b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.780722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-65f3f394-35fe-4ae4-8bb2-23bbcd4a92b3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:08.782481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-65f3f394-35fe-4ae4-8bb2-23bbcd4a92b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:08.798785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c45bf49089848b397e1460c855b0e38', 'request_id': 'req-f95b5999-1abe-45a3-a768-fb459595dd0b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.801960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:08.805250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c45bf49089848b397e1460c855b0e38', 'request_id': 'req-f95b5999-1abe-45a3-a768-fb459595dd0b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.817030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] The token cache is being invalidate because project 1c45bf49089848b397e1460c855b0e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:08.817308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.817945 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f95b5999-1abe-45a3-a768-fb459595dd0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:08.819546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1283/2574] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:08 2022] DELETE /identity/v3/projects/1c45bf49089848b397e1460c855b0e38 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:08.825488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acaa09-d214-4321-b328-6df0309687df None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.826868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acaa09-d214-4321-b328-6df0309687df None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.826868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acaa09-d214-4321-b328-6df0309687df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.841071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1292/2575] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:08.847444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-61acaa09-d214-4321-b328-6df0309687df None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:08.848754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-61acaa09-d214-4321-b328-6df0309687df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:08.849411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.849703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.849961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.856746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1284/2576] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:08.865679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.865956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.866225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.869819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:08.871081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:08.880046 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1293/2577] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:08.884155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:08.887177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:08.888392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:08.893618 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f413-4e5f-489a-a0a0-657ddaca4398 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:08.896275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1285/2578] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:08.898497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:08.899377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.899632 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.899907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.900424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:08.900549 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:08.900855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:08.902502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:08.909757 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc661d45-f78f-46f2-bb5c-82ef1c701003 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:08.914259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:08.914541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1294/2579] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:24:08.914884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.915167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.915447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.916080 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:08.916384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:08.918232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:08.918775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:08.929218 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16840bbc-a9ac-4f13-878a-9a046eb4ebcd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:08.932760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1286/2580] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:24:08.936904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:08.936904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.936904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.936904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.937439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:08.939450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:08.939854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:08.949714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973b7cb-b1f8-4f75-9941-eb2e0b0253ca admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:08.954456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:08.955108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:08.955278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:08.955551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:08.956196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:08.958109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.041937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1295/2581] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:08 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:09.045357 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1287/2582] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:08 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:09.046171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.049265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.110954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75406815-36c6-4df1-8e54-ffdbb1d5f590 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.115389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e115a-6292-4fc0-998f-b0a7edbf7feb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.115690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.116295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.116568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.116832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.117441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.119377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.120247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.120871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.121147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.125488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.125488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.125488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.190894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1296/2583] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:09 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:09.194780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.204436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dec94-0456-4588-908a-4f8b300454cd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.209007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.209578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.209860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.210129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.210803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.212642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.222829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1297/2584] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:24:09.223981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1288/2585] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:09 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:09.226633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.232827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.237408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf972d6-a980-4177-af37-6fc44d4c2974 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.238309 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793876-b370-4206-ad03-7586eb3a5d72 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.241985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.242572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.242805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.242931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.243157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.243475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.243751 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.243880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.244020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.244646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.245365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.246209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.260331 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1289/2586] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:24:09.260556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1298/2587] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:09.261759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.261887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.271506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb93448-9bd9-4aa9-acc5-9a5e647098cf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.272885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf301964-2804-4486-a4b7-6237db6857d5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.276344 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.276709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78093d1c-7533-464a-bdc1-bc28ad6206f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.276918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.277193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.277351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093d1c-7533-464a-bdc1-bc28ad6206f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.277455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.277616 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093d1c-7533-464a-bdc1-bc28ad6206f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.277891 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093d1c-7533-464a-bdc1-bc28ad6206f6 admin admin] PATH_INFO: `/v3/projects/52c00ea9424c40e2a0860d415ee6f502/users/08307786a8ec43f59ae189dab8274b51/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.278068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.279709 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.289613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1299/2588] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:09.293725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.296326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78093d1c-7533-464a-bdc1-bc28ad6206f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c00ea9424c40e2a0860d415ee6f502, user_id=08307786a8ec43f59ae189dab8274b51, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.299014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78093d1c-7533-464a-bdc1-bc28ad6206f6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.305331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e0ab9-1006-4fee-a877-632fa581d8ee admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.312556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea9e0-9892-4de8-be9a-a5dd6765ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.313486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea9e0-9892-4de8-be9a-a5dd6765ffc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.324459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea9e0-9892-4de8-be9a-a5dd6765ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.324459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea9e0-9892-4de8-be9a-a5dd6765ffc8 admin admin] PATH_INFO: `/v3/projects/193f18432ab14088adde9aaf7e4cd8ee/users/2b4671a3cad24fc789812c2197e610b9/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.335122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ea9e0-9892-4de8-be9a-a5dd6765ffc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193f18432ab14088adde9aaf7e4cd8ee, user_id=2b4671a3cad24fc789812c2197e610b9, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.338036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ea9e0-9892-4de8-be9a-a5dd6765ffc8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.410415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1290/2589] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:09 2022] PUT /identity/v3/projects/52c00ea9424c40e2a0860d415ee6f502/users/08307786a8ec43f59ae189dab8274b51/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:09.421185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.421468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.421738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.450074 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] MFA Rules not processed for user `08307786a8ec43f59ae189dab8274b51`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:09.451303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:09.465744 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1300/2590] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:09 2022] PUT /identity/v3/projects/193f18432ab14088adde9aaf7e4cd8ee/users/2b4671a3cad24fc789812c2197e610b9/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:09.484177 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:09.490111 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.490396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.490677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.518349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] MFA Rules not processed for user `2b4671a3cad24fc789812c2197e610b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:09.518896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1291/2591] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:09.519749 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:09.523035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.525014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:09.584013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6bd65-fa78-4b32-bc9f-2e052461375e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.584233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1301/2592] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:09.597721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.599159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:09.643996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.645039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.645361 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.645626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.646268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.648950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.657454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69ad92-0d31-4787-9d21-daf043cc3b9b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.661019 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.661614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.661885 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.662150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.670389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.671888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.680750 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1302/2593] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:24:09.749272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1292/2594] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:09 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:09.757504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.764009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acefa0-765b-4ca3-b3b0-4bf3a8ce6829 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.821962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.822563 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.822881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.823148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.823788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.825677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.943866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1303/2595] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:09 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:09.947966 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.958975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9a3be-ee39-49c3-ba27-cefa9a9200c7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.963768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.964352 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.964631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.965036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.965618 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:09.967244 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:09.978305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1293/2596] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:09.982210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:09.992173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07481ead-bd1a-4d7f-9f77-c14cb7c562cb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:09.996761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7054fe5-d149-43a7-9ecd-351b84909e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:09.997343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7054fe5-d149-43a7-9ecd-351b84909e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:09.997631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7054fe5-d149-43a7-9ecd-351b84909e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:09.997886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7054fe5-d149-43a7-9ecd-351b84909e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:09.998514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7054fe5-d149-43a7-9ecd-351b84909e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:10.000242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7054fe5-d149-43a7-9ecd-351b84909e14 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:10.010273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1304/2597] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:10.014886 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:10.024962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c4796-db37-4c31-aa37-9e7a4cc04dfb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:10.029752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:10.036105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:10.036105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:10.036105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] PATH_INFO: `/v3/projects/278eec6b10654e9d8edf2a29389692d2/users/9ac3b438db8b473092ea8e71b0f0d991/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:10.047957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278eec6b10654e9d8edf2a29389692d2, user_id=9ac3b438db8b473092ea8e71b0f0d991, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:10.050617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:10.149830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1294/2598] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:10 2022] PUT /identity/v3/projects/278eec6b10654e9d8edf2a29389692d2/users/9ac3b438db8b473092ea8e71b0f0d991/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:10.161470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:10.161754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:10.162025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:10.190243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] MFA Rules not processed for user `9ac3b438db8b473092ea8e71b0f0d991`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:10.191518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:10.207826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:10.256112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1305/2599] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:10.260707 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:10.292136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:10.322572 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187ff7d-06cb-476c-8039-e61ba777c226 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:10.327962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:10.328568 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:10.328934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:10.329204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:10.329814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:10.331875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:10.479817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1295/2600] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:10 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:10.483887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:10.494591 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6958e44-0266-4db8-bda9-03ac3f6599be None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:10.553579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:10.554129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:10.554411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:10.554704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:10.555360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:10.557286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:10.667424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1306/2601] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:10 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:10.671902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:10.682673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f80064-06f7-4b6b-bda2-9593f44592c4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:10.687835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:10.688426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:10.688731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:10.689000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:10.689629 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:10.691311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:10.701274 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1296/2602] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:10.705419 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:10.715533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5da17-51cd-4cb9-aec5-a8e9a265819e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:10.720132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:10.720764 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:10.721038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:10.721437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] PATH_INFO: `/v3/projects/9ea2322fd0694cfb9d35bb5d78696e2a/users/0e8ea77c98e54d0ba95be186dd55c92f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:10.738382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea2322fd0694cfb9d35bb5d78696e2a, user_id=0e8ea77c98e54d0ba95be186dd55c92f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:10.739465 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:11.098488 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1307/2603] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:10 2022] PUT /identity/v3/projects/9ea2322fd0694cfb9d35bb5d78696e2a/users/0e8ea77c98e54d0ba95be186dd55c92f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:11.102377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:11.112985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60f996-eeb1-40d0-ab13-1c8b77ac41e3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:11.167916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719574-35e3-493c-a12f-23d8a78c8bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:11.168511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719574-35e3-493c-a12f-23d8a78c8bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:11.168789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719574-35e3-493c-a12f-23d8a78c8bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:11.169059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719574-35e3-493c-a12f-23d8a78c8bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:11.169688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9719574-35e3-493c-a12f-23d8a78c8bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:11.171978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9719574-35e3-493c-a12f-23d8a78c8bb0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:11.181537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1297/2604] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:11.185637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:11.196119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a074b12-4a44-4b55-a853-87db0028ba75 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:11.200627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:11.201266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:11.201544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:11.201812 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] PATH_INFO: `/v3/projects/9ea2322fd0694cfb9d35bb5d78696e2a/users/0e8ea77c98e54d0ba95be186dd55c92f/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:11.206297 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea2322fd0694cfb9d35bb5d78696e2a, user_id=0e8ea77c98e54d0ba95be186dd55c92f, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:11.210088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:11.221855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1308/2605] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:11 2022] PUT /identity/v3/projects/9ea2322fd0694cfb9d35bb5d78696e2a/users/0e8ea77c98e54d0ba95be186dd55c92f/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:11.232498 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:11.232803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:11.233104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:11.261024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] MFA Rules not processed for user `0e8ea77c98e54d0ba95be186dd55c92f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:11.262250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:11.329754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1298/2606] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:11.341884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:11.397524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0a71-6afb-4b1c-b681-afb092c9167d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:11.401211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:11.401805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:11.402084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:11.402353 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:11.411897 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:11.413305 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:11.422893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1309/2607] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:11.586888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:11.597047 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad940c3-edbb-4647-8837-e91950ce7a77 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:11.600600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:11.601184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:11.601464 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:11.601753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:11.622947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:11.659167 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:11.660723 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:11.669409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1299/2608] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:13.270348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:13.271906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:13.285984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:13.329460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5fd6f-237b-48aa-bff6-f7fbc3d51e6e service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:13.333086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:13.333672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:13.333943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:13.334204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:13.336151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:13.393430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:13.394873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:13.403710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1310/2609] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:18.651632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:18.653168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:18.743774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2d2d-3a7c-406d-bd2c-bd68dd1c0a18 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:18.749690 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:18.750487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:18.750875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:18.750875 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:18.752713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:18.766344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:18.768083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba9a0d-fb9f-4fd3-85eb-8aecb42f1fb4 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:18.779197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1300/2610] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:20.853539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:20.925231 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4c1ea-3952-46da-a148-85c9536fd9a4 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:20.929936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:20.930553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:20.930920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:20.931189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] PATH_INFO: `/v3/users/08307786a8ec43f59ae189dab8274b51` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:20.932306 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=08307786a8ec43f59ae189dab8274b51)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:20.934216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:21.115815 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08307786a8ec43f59ae189dab8274b51'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.127484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08307786a8ec43f59ae189dab8274b51'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.131867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:21.134944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08307786a8ec43f59ae189dab8274b51'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.135620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08307786a8ec43f59ae189dab8274b51'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.136213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08307786a8ec43f59ae189dab8274b51'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.209904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1311/2611] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:20 2022] DELETE /identity/v3/users/08307786a8ec43f59ae189dab8274b51 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:21.214752 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:21.215224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:21.215363 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:21.235151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:21.236372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:21.303592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1301/2612] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:21.313461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:21.315114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:21.373630 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87398c9b-2089-4ffb-b222-d4a814147bcc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:21.377184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:21.377773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:21.378041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:21.378304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:21.387652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:21.389043 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:21.399380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1312/2613] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:21.536362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:21.537769 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:21.550223 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cde4d3-4972-4df9-a83d-92fd79abba01 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:21.554851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:21.557396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:21.557396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:21.557396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] PATH_INFO: `/v3/users/2b4671a3cad24fc789812c2197e610b9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:21.557396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4671a3cad24fc789812c2197e610b9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:21.559017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:21.789252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:21.791913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4671a3cad24fc789812c2197e610b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.804017 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4671a3cad24fc789812c2197e610b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.812279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4671a3cad24fc789812c2197e610b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.812279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4671a3cad24fc789812c2197e610b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.812279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d16c13c7-b383-4dad-bc07-0f9045dfc51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4671a3cad24fc789812c2197e610b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.814264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:21.816113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:21.831104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c2a8f-3234-48b8-ab07-263750abbbc0 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:21.835510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:21.836393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:21.836666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:21.836943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] PATH_INFO: `/v3/projects/52c00ea9424c40e2a0860d415ee6f502` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:21.838114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c00ea9424c40e2a0860d415ee6f502)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:21.839966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:21.892081 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1302/2614] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:21 2022] DELETE /identity/v3/users/2b4671a3cad24fc789812c2197e610b9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:21.896404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:21.896691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:21.896970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:21.916631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:21.918884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:21.984856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1303/2615] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:21.988044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c00ea9424c40e2a0860d415ee6f502', 'request_id': 'req-33116bbf-1ce0-4a40-892c-6fe183665968'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:21.995308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:21.996816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:22.055916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c00ea9424c40e2a0860d415ee6f502', 'request_id': 'req-33116bbf-1ce0-4a40-892c-6fe183665968'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.057670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5593b5b-f8ca-4260-b523-041d835f952e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:22.064411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c00ea9424c40e2a0860d415ee6f502', 'request_id': 'req-33116bbf-1ce0-4a40-892c-6fe183665968'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.077076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] The token cache is being invalidate because project 52c00ea9424c40e2a0860d415ee6f502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:22.077516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.078302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.079995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1313/2616] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:21 2022] DELETE /identity/v3/projects/52c00ea9424c40e2a0860d415ee6f502 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:22.084423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:22.088253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:22.117128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:22.117742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:22.117998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:22.118265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:22.120101 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:22.153423 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:22.161729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116bbf-1ce0-4a40-892c-6fe183665968 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:22.166604 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:22.167361 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:22.167672 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:22.167962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] PATH_INFO: `/v3/users/9ac3b438db8b473092ea8e71b0f0d991` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:22.169114 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac3b438db8b473092ea8e71b0f0d991)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:22.171170 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:22.185817 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:22.187545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:22.189172 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:22.201113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1304/2617] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:22.358877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:22.360638 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:22.371521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131020aa-0f17-464e-a58a-d49d09d08b8c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:22.372199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac3b438db8b473092ea8e71b0f0d991'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.375716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:22.376319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:22.376594 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:22.376862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:22.378608 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:22.385051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac3b438db8b473092ea8e71b0f0d991'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.390533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:22.392623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac3b438db8b473092ea8e71b0f0d991'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.393022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:22.393347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac3b438db8b473092ea8e71b0f0d991'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.393960 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac3b438db8b473092ea8e71b0f0d991'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:22.403010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1305/2618] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:24:22.508508 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1314/2619] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:22 2022] DELETE /identity/v3/users/9ac3b438db8b473092ea8e71b0f0d991 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:22.989303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.001765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c7a97-e782-4bde-9d13-07d052194bbb service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.064020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.064644 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.064917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.065184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] PATH_INFO: `/v3/projects/278eec6b10654e9d8edf2a29389692d2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.066424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278eec6b10654e9d8edf2a29389692d2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.068271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.153432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.154975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:23.168307 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8013895-7f8e-49e3-825b-464475052393 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.173358 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.174055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.174331 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.174600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] PATH_INFO: `/v3/projects/193f18432ab14088adde9aaf7e4cd8ee` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.175788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193f18432ab14088adde9aaf7e4cd8ee)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.177729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.231392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278eec6b10654e9d8edf2a29389692d2', 'request_id': 'req-4a5c7604-379d-413b-afac-29baf4614bc4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.304118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278eec6b10654e9d8edf2a29389692d2', 'request_id': 'req-4a5c7604-379d-413b-afac-29baf4614bc4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.311436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278eec6b10654e9d8edf2a29389692d2', 'request_id': 'req-4a5c7604-379d-413b-afac-29baf4614bc4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.326762 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193f18432ab14088adde9aaf7e4cd8ee', 'request_id': 'req-b6386ab6-bc70-4929-974d-fae48e581486'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.328486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] The token cache is being invalidate because project 278eec6b10654e9d8edf2a29389692d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:23.333921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.334730 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.336840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1306/2620] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:22 2022] DELETE /identity/v3/projects/278eec6b10654e9d8edf2a29389692d2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:23.340621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.341970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:23.392714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:23.412362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c7604-379d-413b-afac-29baf4614bc4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.429246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193f18432ab14088adde9aaf7e4cd8ee', 'request_id': 'req-b6386ab6-bc70-4929-974d-fae48e581486'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.436580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193f18432ab14088adde9aaf7e4cd8ee', 'request_id': 'req-b6386ab6-bc70-4929-974d-fae48e581486'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.451904 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] The token cache is being invalidate because project 193f18432ab14088adde9aaf7e4cd8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:23.452195 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.453703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b6386ab6-bc70-4929-974d-fae48e581486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.461398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1315/2621] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:23 2022] DELETE /identity/v3/projects/193f18432ab14088adde9aaf7e4cd8ee => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:23.462824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aee2f2-ec10-41ae-ad57-4d7b2e4f2a3a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.463113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aee2f2-ec10-41ae-ad57-4d7b2e4f2a3a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.463385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aee2f2-ec10-41ae-ad57-4d7b2e4f2a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.478116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.478842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.479072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.479303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] PATH_INFO: `/v3/users/0e8ea77c98e54d0ba95be186dd55c92f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.480488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8ea77c98e54d0ba95be186dd55c92f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.482486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.485394 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-15aee2f2-ec10-41ae-ad57-4d7b2e4f2a3a None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:23.486787 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-15aee2f2-ec10-41ae-ad57-4d7b2e4f2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:23.495679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1316/2622] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:23.503373 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.503766 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.504423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.524738 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:23.525951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:23.534636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1317/2623] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:23.538793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.547916 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76ff83-68a4-471f-ad5a-650ad4025e63 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.553620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.554210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.554486 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.554781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.555426 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:23.555723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.557312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.569429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1318/2624] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:24:23.573753 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.584634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966a4f6-f1c9-4dbe-94dd-d10dfa701004 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.589873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.590469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.590793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.591068 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.591689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.593758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.693739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8ea77c98e54d0ba95be186dd55c92f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.705596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8ea77c98e54d0ba95be186dd55c92f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.713621 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8ea77c98e54d0ba95be186dd55c92f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.719316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8ea77c98e54d0ba95be186dd55c92f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.719316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8ea77c98e54d0ba95be186dd55c92f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:23.719561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1319/2625] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:23 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:23.723147 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.725081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:23.760269 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:23.796355 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6a37f-01ac-4f62-be68-4f26e5464de3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.803884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.808622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.808622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.808622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.808622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.808622 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.834359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1307/2626] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:23 2022] DELETE /identity/v3/users/0e8ea77c98e54d0ba95be186dd55c92f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:23.888390 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1320/2627] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:23 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:23.892380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:23.972785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc968a15-44f7-4681-a9c8-6b66185566a3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:23.978387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:23.979286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:23.979676 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:23.980051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:23.981181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:23.983839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:23.996617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1308/2628] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:24:24.001314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:24.012151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39736d-2caf-4705-bc1c-f8c71a61980f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:24.017714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc5c09-3804-463f-b74d-47a966b0b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:24.018466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5c09-3804-463f-b74d-47a966b0b37e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:24.018831 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5c09-3804-463f-b74d-47a966b0b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:24.019185 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5c09-3804-463f-b74d-47a966b0b37e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:24.019911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc5c09-3804-463f-b74d-47a966b0b37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:24.021674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc5c09-3804-463f-b74d-47a966b0b37e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:24.034250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1321/2629] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:24.038418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:24.050471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788c409-65b3-463e-acdb-7679d55e312b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:24.059533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:24.060492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:24.060878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:24.061259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] PATH_INFO: `/v3/projects/860550889509456e867356246184afa5/users/e4346634525f46e3ba69da7ee990b786/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:24.064956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:24.088666 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860550889509456e867356246184afa5, user_id=e4346634525f46e3ba69da7ee990b786, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:24.091826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:24.211646 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1309/2630] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:24 2022] PUT /identity/v3/projects/860550889509456e867356246184afa5/users/e4346634525f46e3ba69da7ee990b786/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:24.223011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:24.223568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:24.223670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:24.254401 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] MFA Rules not processed for user `e4346634525f46e3ba69da7ee990b786`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:24.255623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:24.342785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1322/2631] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:24.357728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:24.359590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:24.465114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0a75-540e-46bc-bb5b-da19423eb2f0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:24.469266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281f76a-ad7c-411a-bb7c-55d5eb72302a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:24.469889 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281f76a-ad7c-411a-bb7c-55d5eb72302a service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:24.470164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281f76a-ad7c-411a-bb7c-55d5eb72302a service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:24.470431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281f76a-ad7c-411a-bb7c-55d5eb72302a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:24.480388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281f76a-ad7c-411a-bb7c-55d5eb72302a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:24.481901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281f76a-ad7c-411a-bb7c-55d5eb72302a service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:24.497856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1310/2632] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:24.734806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:24.743374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:24.841411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205560-6a96-4e31-a3a3-740fde39a0ba None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:24.846931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:24.847643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:24.847934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:24.848199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] PATH_INFO: `/v3/projects/9ea2322fd0694cfb9d35bb5d78696e2a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:24.849329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea2322fd0694cfb9d35bb5d78696e2a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:24.851810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:25.053350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:25.061472 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea2322fd0694cfb9d35bb5d78696e2a', 'request_id': 'req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:25.187078 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea2322fd0694cfb9d35bb5d78696e2a', 'request_id': 'req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:25.194134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea2322fd0694cfb9d35bb5d78696e2a', 'request_id': 'req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:25.211086 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] The token cache is being invalidate because project 9ea2322fd0694cfb9d35bb5d78696e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:25.211517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:25.212235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9b4f16b2-de3b-40a7-9908-6dbab0c2ab46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:25.214187 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1323/2633] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:24 2022] DELETE /identity/v3/projects/9ea2322fd0694cfb9d35bb5d78696e2a => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:25.222028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:25.222633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:25.223067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:25.259061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:25.260303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:25.335045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1311/2634] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:25.345658 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:25.349674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:25.349674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:25.369321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:25.370647 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:25.385483 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1324/2635] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:25.389831 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:25.400100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d486d8-b83f-4801-ab12-599d1f408897 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:25.404713 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:25.405300 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:25.405573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:25.405839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:25.408124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:25.408124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:25.408933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:25.420732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1312/2636] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:24:25.432323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:25.441663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f59c6-5925-4267-8c1f-ba907a21d348 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:25.446237 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:25.446861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:25.447142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:25.447408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:25.448030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:25.449900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:25.607928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1325/2637] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:25 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:25.612119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:25.622303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bb77d-5f95-4b28-813d-4352c7ca0ce6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:25.687755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:25.688369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:25.688657 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:25.688934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:25.689573 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:25.691538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:26.027027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1313/2638] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:25 2022] POST /identity/v3/users => generated 288 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:26.031646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:26.041652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac3f6a-e70e-415c-9c9e-f49e78761c27 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:26.046168 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:26.047478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:26.047763 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:26.048027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:26.048653 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:26.050221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:26.060596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1326/2639] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:24:26.067756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:26.072932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:26.078037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cb230-f877-4305-b0a3-9b523f22b88f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:26.082465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa6fc7-ad34-4023-96dc-872430d75a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:26.083315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa6fc7-ad34-4023-96dc-872430d75a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:26.083595 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa6fc7-ad34-4023-96dc-872430d75a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:26.083864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa6fc7-ad34-4023-96dc-872430d75a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:26.084500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa6fc7-ad34-4023-96dc-872430d75a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:26.086346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa6fc7-ad34-4023-96dc-872430d75a52 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:26.099041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1314/2640] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:26.103316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:26.114177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec99db-1a0e-435e-9632-1f2321f4affc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:26.121460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:26.121460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:26.121460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:26.121460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] PATH_INFO: `/v3/projects/fc21d5e1bdd341119331c2433a56a0ab/users/42ef17127eea4031804246187b121b83/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:26.134708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc21d5e1bdd341119331c2433a56a0ab, user_id=42ef17127eea4031804246187b121b83, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:26.137323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:26.590982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1327/2641] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:26 2022] PUT /identity/v3/projects/fc21d5e1bdd341119331c2433a56a0ab/users/42ef17127eea4031804246187b121b83/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:26.607028 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:26.607312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:26.607587 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:26.642133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] MFA Rules not processed for user `42ef17127eea4031804246187b121b83`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:26.643533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:26.720016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1315/2642] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:26.724746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:26.794054 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfb4fa-0159-4b41-b5af-87609ce6ee7d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:26.799298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:26.799925 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:26.800210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:26.800489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:26.801122 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:26.803040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:26.964016 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1328/2643] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:26 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:26.968246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:26.979910 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee9d7d-76a7-4c1f-967e-7986e9feb62f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:27.039197 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:27.039810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:27.040088 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:27.040362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:27.040989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:27.042919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:27.164986 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1316/2644] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:26 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:27.166886 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:27.182467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcb1e5-eadc-48a8-809c-b91ede8d27c5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:27.195200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:27.195200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:27.195200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:27.195200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:27.195625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:27.195625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:27.201975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1329/2645] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:27.206459 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:27.217195 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de87ed-956c-4405-8502-627fcf34149a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:27.222019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abdfe6-eccd-4f5c-b2b1-0ec952ef590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:27.222620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abdfe6-eccd-4f5c-b2b1-0ec952ef590e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:27.222934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abdfe6-eccd-4f5c-b2b1-0ec952ef590e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:27.223201 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abdfe6-eccd-4f5c-b2b1-0ec952ef590e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:27.223871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abdfe6-eccd-4f5c-b2b1-0ec952ef590e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:27.225623 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abdfe6-eccd-4f5c-b2b1-0ec952ef590e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:27.235917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1317/2646] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:27.240705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:27.250422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fdaa7-b061-4ea7-94ff-81012ca80b24 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:27.256189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:27.257123 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:27.257564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:27.257985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] PATH_INFO: `/v3/projects/e831c1aaddff4f6fb856d6d62bff8891/users/23f9dbb2263c48e79c7ce4a29a8b7705/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:27.280914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e831c1aaddff4f6fb856d6d62bff8891, user_id=23f9dbb2263c48e79c7ce4a29a8b7705, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:27.280914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:27.572966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1330/2647] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:27 2022] PUT /identity/v3/projects/e831c1aaddff4f6fb856d6d62bff8891/users/23f9dbb2263c48e79c7ce4a29a8b7705/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:27.584632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:27.584932 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:27.585353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:27.625725 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] MFA Rules not processed for user `23f9dbb2263c48e79c7ce4a29a8b7705`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:27.627077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:27.695921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1318/2648] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:27.700850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:27.767340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2a9a8-4899-44ed-8977-34a86d5ec79b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:27.772798 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:27.773416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:27.773694 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:27.773963 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:27.774580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:27.776730 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:27.925734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1331/2649] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:27 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:27.933329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:27.951781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611629cf-d4ce-4103-9072-0fa66845c79f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:27.963023 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:27.997954 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:28.015692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:28.016241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.016693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.016992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.017742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:28.021150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:28.125942 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:28.132058 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1319/2650] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:27 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:28.141226 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f1f11-62d4-430b-b135-88b7cfdcc0c9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:28.146075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:28.146758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.147035 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.147308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.148270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:28.149911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:28.165546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1332/2651] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:24:28.169824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:28.180844 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8354c69-cb95-40cf-afb9-ceabdce0ab25 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:28.185988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:28.186558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.186863 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.187128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] PATH_INFO: `/v3/projects/81039320df3f4d42bab6d62d9cab3773/users/2743844143eb4997b19ee2e0b308475d/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.202919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=81039320df3f4d42bab6d62d9cab3773, user_id=2743844143eb4997b19ee2e0b308475d, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:28.206606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:28.475803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1320/2652] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:28 2022] PUT /identity/v3/projects/81039320df3f4d42bab6d62d9cab3773/users/2743844143eb4997b19ee2e0b308475d/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:28.479708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:28.494432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ab65-fad1-43a8-9b21-df649b93514f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:28.546312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b60943-2d49-49e9-acd1-638312f5f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:28.551487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b60943-2d49-49e9-acd1-638312f5f063 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.551487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b60943-2d49-49e9-acd1-638312f5f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.551487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b60943-2d49-49e9-acd1-638312f5f063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.551487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b60943-2d49-49e9-acd1-638312f5f063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:28.551487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b60943-2d49-49e9-acd1-638312f5f063 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:28.560061 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1333/2653] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:28.564205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:28.575603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99b4a4-e713-4ae0-ab59-50eeab8e41ad admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:28.580295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:28.580936 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.581739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.582014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] PATH_INFO: `/v3/projects/81039320df3f4d42bab6d62d9cab3773/users/2743844143eb4997b19ee2e0b308475d/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.583917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=81039320df3f4d42bab6d62d9cab3773, user_id=2743844143eb4997b19ee2e0b308475d, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:28.586839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:28.598499 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1321/2654] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:28 2022] PUT /identity/v3/projects/81039320df3f4d42bab6d62d9cab3773/users/2743844143eb4997b19ee2e0b308475d/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:28.665901 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.666189 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.666461 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.699813 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] MFA Rules not processed for user `2743844143eb4997b19ee2e0b308475d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:28.701079 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:28.780936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1334/2655] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:28.794539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:28.796918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:28.867040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c828d3-1232-4926-8232-b17f2bdde1ba admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:28.870938 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:28.871714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:28.872034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:28.872749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:28.944771 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:28.946333 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:28.969827 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1322/2656] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:29.114001 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:31.815328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:31.816890 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:31.827692 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:31.883772 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ceb6-8aa3-4810-ac1b-8df62bcc23ea None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:31.888342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:31.888979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:31.889256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:31.889524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] PATH_INFO: `/v3/users/d0a56142f0fc4c32b7e34ec74803407f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:31.890684 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a56142f0fc4c32b7e34ec74803407f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:31.892600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:32.088129 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a56142f0fc4c32b7e34ec74803407f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.106726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a56142f0fc4c32b7e34ec74803407f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.114325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a56142f0fc4c32b7e34ec74803407f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.115208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a56142f0fc4c32b7e34ec74803407f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.115846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a56142f0fc4c32b7e34ec74803407f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.207121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1335/2657] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:31 2022] DELETE /identity/v3/users/d0a56142f0fc4c32b7e34ec74803407f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:32.483253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:32.488546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:32.490094 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:32.495474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52692dc-d924-48cf-8c75-34542d76d656 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:32.578382 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bc261-0b86-44fc-ba2f-522cf9c312ab admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:32.583184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:32.584174 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:32.584447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:32.584714 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:32.586849 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:32.632962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:32.652543 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:32.653988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:32.664835 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1336/2658] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:32.686491 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:32.687157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:32.687431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:32.687699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] PATH_INFO: `/v3/projects/e855cc11f6a142f9a6d31088812beeef` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:32.689129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e855cc11f6a142f9a6d31088812beeef)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:32.691113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:32.815087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e855cc11f6a142f9a6d31088812beeef', 'request_id': 'req-b9759936-12a1-4109-88b6-91a09b1088e0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.903365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e855cc11f6a142f9a6d31088812beeef', 'request_id': 'req-b9759936-12a1-4109-88b6-91a09b1088e0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.910953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e855cc11f6a142f9a6d31088812beeef', 'request_id': 'req-b9759936-12a1-4109-88b6-91a09b1088e0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.923956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] The token cache is being invalidate because project e855cc11f6a142f9a6d31088812beeef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:32.924296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.924995 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:32.927617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1323/2659] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:32 2022] DELETE /identity/v3/projects/e855cc11f6a142f9a6d31088812beeef => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:32.931711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:32.933091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:32.947880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfd4d8-20b4-41aa-8501-dbe09833416c service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:33.007833 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:33.008491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:33.008768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:33.009036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] PATH_INFO: `/v3/users/d5903cbd5bb646a7acf4c5a553cbe1f5` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:33.010265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5903cbd5bb646a7acf4c5a553cbe1f5)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:33.012097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:33.232115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5903cbd5bb646a7acf4c5a553cbe1f5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.254434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5903cbd5bb646a7acf4c5a553cbe1f5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.261967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5903cbd5bb646a7acf4c5a553cbe1f5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.262747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5903cbd5bb646a7acf4c5a553cbe1f5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.263365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5903cbd5bb646a7acf4c5a553cbe1f5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.389372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1337/2660] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:32 2022] DELETE /identity/v3/users/d5903cbd5bb646a7acf4c5a553cbe1f5 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:33.635141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:33.636732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:33.653386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9759936-12a1-4109-88b6-91a09b1088e0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:33.714014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:33.714675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:33.714963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:33.715297 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] PATH_INFO: `/v3/projects/3cf92ba7f2944bb89fafda9d273eedac` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:33.716511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf92ba7f2944bb89fafda9d273eedac)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:33.718631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:33.823418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:33.838296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf92ba7f2944bb89fafda9d273eedac', 'request_id': 'req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.907301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf92ba7f2944bb89fafda9d273eedac', 'request_id': 'req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.913134 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf92ba7f2944bb89fafda9d273eedac', 'request_id': 'req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.924397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] The token cache is being invalidate because project 3cf92ba7f2944bb89fafda9d273eedac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:33.924688 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.925346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:33.927473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1324/2661] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:33 2022] DELETE /identity/v3/projects/3cf92ba7f2944bb89fafda9d273eedac => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:34.191474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:34.193006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:34.253240 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42eede-2f11-48df-9771-e781899bb679 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:34.267209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:34.306936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:34.310861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:34.310861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:34.310861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:34.310861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:34.375137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:34.376634 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5a6a8-599b-4386-af78-1b61c238bd43 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:34.386430 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1338/2662] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:40.072478 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:40.073965 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:40.137462 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a3ed-483e-4cc1-aa78-e472cb0ced19 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:40.141918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:40.142527 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:40.142830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:40.143100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] PATH_INFO: `/v3/users/42ef17127eea4031804246187b121b83` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:40.144214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ef17127eea4031804246187b121b83)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:40.146125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:40.371742 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ef17127eea4031804246187b121b83'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:40.383268 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ef17127eea4031804246187b121b83'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:40.390103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ef17127eea4031804246187b121b83'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:40.390731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ef17127eea4031804246187b121b83'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:40.391323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ef17127eea4031804246187b121b83'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:40.480786 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1325/2663] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:40 2022] DELETE /identity/v3/users/42ef17127eea4031804246187b121b83 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:40.485207 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:40.485680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:40.485803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:40.504495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:40.505739 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:40.564442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1339/2664] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:40.574253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:40.575675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:40.618935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:40.634835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4f3c2-ac4e-455f-a250-03af27b5c896 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:40.638341 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:40.638975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:40.639252 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:40.639521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:40.649071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:40.650564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:40.660734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1326/2665] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:41.119509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:41.121280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:41.136310 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf90a03-c913-423d-9710-688b0c39f858 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:41.141027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:41.141559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:41.141834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:41.146945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] PATH_INFO: `/v3/projects/fc21d5e1bdd341119331c2433a56a0ab` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:41.146945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc21d5e1bdd341119331c2433a56a0ab)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:41.146945 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:41.279837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc21d5e1bdd341119331c2433a56a0ab', 'request_id': 'req-2bbb6c87-3c8b-4d58-94fe-187b03844f66'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.434152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc21d5e1bdd341119331c2433a56a0ab', 'request_id': 'req-2bbb6c87-3c8b-4d58-94fe-187b03844f66'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.438481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:41.440765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc21d5e1bdd341119331c2433a56a0ab', 'request_id': 'req-2bbb6c87-3c8b-4d58-94fe-187b03844f66'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.452434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] The token cache is being invalidate because project fc21d5e1bdd341119331c2433a56a0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:41.452719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.453354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.454858 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1340/2666] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:41 2022] DELETE /identity/v3/projects/fc21d5e1bdd341119331c2433a56a0ab => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:41.458868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:41.468819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759a07-f58f-4f41-9be7-be2220b9453b service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:41.526860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:41.527479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:41.527825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:41.528093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] PATH_INFO: `/v3/users/23f9dbb2263c48e79c7ce4a29a8b7705` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:41.529185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f9dbb2263c48e79c7ce4a29a8b7705)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:41.531159 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:41.720745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f9dbb2263c48e79c7ce4a29a8b7705'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.732917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f9dbb2263c48e79c7ce4a29a8b7705'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.740281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f9dbb2263c48e79c7ce4a29a8b7705'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.740946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f9dbb2263c48e79c7ce4a29a8b7705'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.741583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f9dbb2263c48e79c7ce4a29a8b7705'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:41.832231 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1327/2667] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:41 2022] DELETE /identity/v3/users/23f9dbb2263c48e79c7ce4a29a8b7705 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:42.235768 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:42.237316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:42.253051 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb6c87-3c8b-4d58-94fe-187b03844f66 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:42.313018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:42.313248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:42.313499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:42.317516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] PATH_INFO: `/v3/projects/e831c1aaddff4f6fb856d6d62bff8891` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:42.317516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e831c1aaddff4f6fb856d6d62bff8891)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:42.317516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:42.522040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e831c1aaddff4f6fb856d6d62bff8891', 'request_id': 'req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.591979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e831c1aaddff4f6fb856d6d62bff8891', 'request_id': 'req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.598081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e831c1aaddff4f6fb856d6d62bff8891', 'request_id': 'req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.609011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] The token cache is being invalidate because project e831c1aaddff4f6fb856d6d62bff8891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:42.609295 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.609924 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.611429 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1341/2668] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:42 2022] DELETE /identity/v3/projects/e831c1aaddff4f6fb856d6d62bff8891 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:42.615601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:42.617051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:42.630305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c0a7-ce08-4342-8aae-2588fd054933 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:42.680849 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:42.686471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:42.687129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:42.687404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:42.687673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] PATH_INFO: `/v3/users/2743844143eb4997b19ee2e0b308475d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:42.688815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2743844143eb4997b19ee2e0b308475d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:42.690732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:42.873799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2743844143eb4997b19ee2e0b308475d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.889761 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2743844143eb4997b19ee2e0b308475d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.896859 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2743844143eb4997b19ee2e0b308475d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.897634 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2743844143eb4997b19ee2e0b308475d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:42.898255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4f42215a-65d0-455e-901f-2b4ae2985729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2743844143eb4997b19ee2e0b308475d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:43.009465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1328/2669] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:42 2022] DELETE /identity/v3/users/2743844143eb4997b19ee2e0b308475d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:43.436214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:43.437635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:43.450205 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4ab1-85eb-4403-ba01-ba24e5eb0e64 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:43.520854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:43.520854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:43.521610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:43.521610 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] PATH_INFO: `/v3/projects/81039320df3f4d42bab6d62d9cab3773` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:43.522872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81039320df3f4d42bab6d62d9cab3773)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:43.524878 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:43.547693 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:43.631184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81039320df3f4d42bab6d62d9cab3773', 'request_id': 'req-04287924-d5ac-4e94-8152-9a437ef14af7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:43.694612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81039320df3f4d42bab6d62d9cab3773', 'request_id': 'req-04287924-d5ac-4e94-8152-9a437ef14af7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:43.700279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81039320df3f4d42bab6d62d9cab3773', 'request_id': 'req-04287924-d5ac-4e94-8152-9a437ef14af7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:43.712774 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] The token cache is being invalidate because project 81039320df3f4d42bab6d62d9cab3773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:43.713064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:43.713705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-04287924-d5ac-4e94-8152-9a437ef14af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:43.715210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1342/2670] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:43 2022] DELETE /identity/v3/projects/81039320df3f4d42bab6d62d9cab3773 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:43.721556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:43.722039 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:43.722140 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:43.741856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:43.743092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:43.803409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1329/2671] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:43.810860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:43.811260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:43.816917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:43.831463 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:43.833447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:43.841674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1343/2672] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:43.846199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:43.856188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1c13-7f6f-4820-bc34-463d6c701ae9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:43.860839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:43.861430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:43.861711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:43.861979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:43.862617 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:43.862949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:43.864520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:43.876501 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1330/2673] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:24:43.881489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:43.890724 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cbf52-2143-4753-94ac-915ad31e5ba2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:43.895990 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:43.896572 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:43.896845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:43.897112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:43.897718 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:43.899633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:43.987595 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1344/2674] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:43 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:43.991815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:44.001746 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237675-e18b-44fc-8172-c79c65aef710 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:44.059578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:44.060175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:44.060455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:44.060728 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:44.061343 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:44.063293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:44.073825 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:44.197832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1331/2675] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:43 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:44.201974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:44.212512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65627b-22a8-40d2-a62f-96ea3b8718c2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:44.217191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:44.217781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:44.218060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:44.218329 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:44.219325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:44.220655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:44.230455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1345/2676] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:44.234386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:44.244946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612b43-599c-49ec-9594-435aecca8773 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:44.249865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5f097-e3a9-4454-8d47-5b7855510273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:44.254290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f097-e3a9-4454-8d47-5b7855510273 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:44.254290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f097-e3a9-4454-8d47-5b7855510273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:44.254290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f097-e3a9-4454-8d47-5b7855510273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:44.254290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5f097-e3a9-4454-8d47-5b7855510273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:44.254290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5f097-e3a9-4454-8d47-5b7855510273 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:44.264156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1332/2677] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:44.268220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:44.278115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af39d-a075-4157-b522-5652c9170359 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:44.282468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:44.283130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:44.283412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:44.283707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] PATH_INFO: `/v3/projects/9555de757fbc4bfc8c29ac242e9498b4/users/8f6181e13d3c4b0f877b2baad953b30e/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:44.298455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9555de757fbc4bfc8c29ac242e9498b4, user_id=8f6181e13d3c4b0f877b2baad953b30e, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:44.301723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:44.366048 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1346/2678] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:44 2022] PUT /identity/v3/projects/9555de757fbc4bfc8c29ac242e9498b4/users/8f6181e13d3c4b0f877b2baad953b30e/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:44.377758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:44.378042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:44.378314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:44.405762 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] MFA Rules not processed for user `8f6181e13d3c4b0f877b2baad953b30e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:44.407011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:44.469684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1333/2679] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:44.481014 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:44.482770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:44.539887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a81425-eb25-4f08-8166-dc7aee167d9c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:44.543554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1de40-4cb3-4e1d-ae68-0fb5b30a0098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:44.544151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1de40-4cb3-4e1d-ae68-0fb5b30a0098 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:44.544434 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1de40-4cb3-4e1d-ae68-0fb5b30a0098 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:44.544689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1de40-4cb3-4e1d-ae68-0fb5b30a0098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:44.554405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1de40-4cb3-4e1d-ae68-0fb5b30a0098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:44.555861 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1de40-4cb3-4e1d-ae68-0fb5b30a0098 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:44.565656 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1347/2680] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:48.892173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:48.955007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9ad6d-fd71-4f3c-94e4-d2c5a321e6b7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:48.960187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:48.960807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:48.961083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:48.961350 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] PATH_INFO: `/v3/users/8f6181e13d3c4b0f877b2baad953b30e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:48.962460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6181e13d3c4b0f877b2baad953b30e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:48.964398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:49.196011 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6181e13d3c4b0f877b2baad953b30e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:49.208852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6181e13d3c4b0f877b2baad953b30e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:49.215993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6181e13d3c4b0f877b2baad953b30e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:49.216685 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6181e13d3c4b0f877b2baad953b30e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:49.217289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6181e13d3c4b0f877b2baad953b30e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:49.218972 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:49.331426 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1334/2681] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:24:48 2022] DELETE /identity/v3/users/8f6181e13d3c4b0f877b2baad953b30e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:49.335943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:49.336439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:49.336547 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:49.354816 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:49.356002 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:49.399505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:49.414208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1348/2682] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:49.424124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:49.426133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:49.483851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3ef2b-2d4d-491d-96e3-6741bbee1453 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:49.487449 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69496f-f032-4470-b36c-db0c2ee87e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:49.488540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69496f-f032-4470-b36c-db0c2ee87e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:49.488815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69496f-f032-4470-b36c-db0c2ee87e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:49.489083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69496f-f032-4470-b36c-db0c2ee87e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:49.498012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69496f-f032-4470-b36c-db0c2ee87e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:49.499487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69496f-f032-4470-b36c-db0c2ee87e09 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:49.509774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1335/2683] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:49.979999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:49.981490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:49.999142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a2ab3-3842-4609-9bbb-ff259fcd3460 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.006356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.006356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.006356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.006356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] PATH_INFO: `/v3/projects/9555de757fbc4bfc8c29ac242e9498b4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.007285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9555de757fbc4bfc8c29ac242e9498b4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.008624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.111438 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9555de757fbc4bfc8c29ac242e9498b4', 'request_id': 'req-8b69fe38-7af1-47c1-90ed-609bd6cf7040'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:50.205208 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9555de757fbc4bfc8c29ac242e9498b4', 'request_id': 'req-8b69fe38-7af1-47c1-90ed-609bd6cf7040'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:50.212145 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9555de757fbc4bfc8c29ac242e9498b4', 'request_id': 'req-8b69fe38-7af1-47c1-90ed-609bd6cf7040'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:50.223832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] The token cache is being invalidate because project 9555de757fbc4bfc8c29ac242e9498b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:24:50.224120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:50.224755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b69fe38-7af1-47c1-90ed-609bd6cf7040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:24:50.226360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1349/2684] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:24:49 2022] DELETE /identity/v3/projects/9555de757fbc4bfc8c29ac242e9498b4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:50.231677 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.233781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.233781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.252257 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:50.254030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:50.313481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1336/2685] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:50.321504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.321789 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.322062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.341966 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:50.343193 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:50.351541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1350/2686] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:50.355934 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.365428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55221e7e-38de-4c85-a680-0fb881ebcdbb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.369808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.370381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.370678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.370939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.371565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:24:50.371857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.373533 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.384790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1337/2687] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:24:50.392044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.401680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1abdf-251a-4e9f-9c64-418bda4fb80b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.406348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.406970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.407248 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.407517 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.408146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.410609 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.490872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1351/2688] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:50 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:24:50.495181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.505203 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d42fe-85c9-4d71-bce0-e755638e3696 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.560651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.561241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.561514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.561782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.562398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.564338 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.649714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1338/2689] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:50 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:50.653940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.664232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717217a-ad3b-4350-84d7-247c350b5e79 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.668824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.669407 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.669683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.669948 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.670575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.672211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.682040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1352/2690] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:50.686086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.696710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31126c96-e3bb-441e-b2cf-9f123764fd05 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.701394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a475-44f0-415b-94c7-31537c8bcabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.701989 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a475-44f0-415b-94c7-31537c8bcabf admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.702279 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a475-44f0-415b-94c7-31537c8bcabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.702535 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a475-44f0-415b-94c7-31537c8bcabf admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.703208 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a475-44f0-415b-94c7-31537c8bcabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.704824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a475-44f0-415b-94c7-31537c8bcabf admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.710954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:50.716460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1339/2691] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:50.720560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.730278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39b6ba-c2e3-4b2b-801a-a35f7af931f5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:50.735183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:50.735826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.736100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.736369 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] PATH_INFO: `/v3/projects/8d2bfa865669477f8757e977da29420a/users/05e18a4fd73e435aa6d0f10a37405698/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.750799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2bfa865669477f8757e977da29420a, user_id=05e18a4fd73e435aa6d0f10a37405698, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:50.753862 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:50.843723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1353/2692] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:50 2022] PUT /identity/v3/projects/8d2bfa865669477f8757e977da29420a/users/05e18a4fd73e435aa6d0f10a37405698/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:24:50.850941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c5a07-475c-45b4-ba2d-801b2534c1bb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:50.851230 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c5a07-475c-45b4-ba2d-801b2534c1bb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:50.851504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c5a07-475c-45b4-ba2d-801b2534c1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:50.871040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f23c5a07-475c-45b4-ba2d-801b2534c1bb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:50.872266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f23c5a07-475c-45b4-ba2d-801b2534c1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:50.930904 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1340/2693] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:24:50.940716 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:50.943097 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:51.007781 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17899b80-9cae-43ff-b183-d26550901bc7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:51.011482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:51.012075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:51.012348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:51.012613 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:51.021747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:51.023199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:51.032413 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1354/2694] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:24:56.497540 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:56.498107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:56.498107 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:56.530034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] MFA Rules not processed for user `05e18a4fd73e435aa6d0f10a37405698`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:24:56.531795 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:24:56.619758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1341/2695] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:24:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:24:56.622119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:56.634470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513702d-a3b1-4de5-a412-6fb710c7d982 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:56.639947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:56.640747 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:56.640956 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:56.641264 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:56.641707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:56.645931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:56.656062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:56.748743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1355/2696] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:24:56 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:56.752896 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:56.764342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163003b-8c4d-4df9-a0f0-a3f021d79753 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:56.820661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:56.821255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:56.821539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:56.821810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:56.822505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:56.824419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:56.916521 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1342/2697] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:24:56 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:24:56.918112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:56.929203 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874bd79-a111-4264-88a8-c1c73bd400f3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:56.933785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:56.934379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:56.934683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:56.934985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:56.935624 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:56.937201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:56.947580 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1356/2698] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:56.951758 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:56.962109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13d90d-80ba-43ac-8785-7e9246787f44 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:56.966622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:56.967346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:56.967718 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:56.967979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] PATH_INFO: `/v3/projects/bfbaff8da48c497e8c8d04905de027be/users/6d8cf8de7c644af78e059460ff804840/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:56.981993 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbaff8da48c497e8c8d04905de027be, user_id=6d8cf8de7c644af78e059460ff804840, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:56.984777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:56.988209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:24:57.091692 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1343/2699] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:56 2022] PUT /identity/v3/projects/bfbaff8da48c497e8c8d04905de027be/users/6d8cf8de7c644af78e059460ff804840/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:24:57.095958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:57.106104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b4090-9c97-4f41-ae1a-bb563743fb2d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:57.169765 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3aedd-2d20-4495-b2f1-92375e005559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:57.170392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3aedd-2d20-4495-b2f1-92375e005559 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:57.170810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3aedd-2d20-4495-b2f1-92375e005559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:57.170939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3aedd-2d20-4495-b2f1-92375e005559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:57.171579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3aedd-2d20-4495-b2f1-92375e005559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:57.173172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3aedd-2d20-4495-b2f1-92375e005559 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:57.184970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1357/2700] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:24:57.190577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:24:57.203246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb4f26-d3f3-4148-88b1-1288e17f8615 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:24:57.209076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:24:57.210085 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:24:57.210364 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:24:57.210636 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] PATH_INFO: `/v3/projects/bfbaff8da48c497e8c8d04905de027be/users/6d8cf8de7c644af78e059460ff804840/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:24:57.212613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbaff8da48c497e8c8d04905de027be, user_id=6d8cf8de7c644af78e059460ff804840, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:24:57.215289 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:24:57.226719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1344/2701] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:24:57 2022] PUT /identity/v3/projects/bfbaff8da48c497e8c8d04905de027be/users/6d8cf8de7c644af78e059460ff804840/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:03.270132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:03.270803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:03.270803 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:03.325657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] MFA Rules not processed for user `6d8cf8de7c644af78e059460ff804840`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:03.327003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:03.408561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1358/2702] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:03.428610 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:03.430976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:03.519571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da068c-95ca-4bdd-a6d7-e8cd514e1761 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:03.525245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336ae55-cc42-431d-a27f-9417a6c581d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:03.525901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336ae55-cc42-431d-a27f-9417a6c581d0 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:03.526207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336ae55-cc42-431d-a27f-9417a6c581d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:03.527592 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336ae55-cc42-431d-a27f-9417a6c581d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:03.590086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336ae55-cc42-431d-a27f-9417a6c581d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:03.593115 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336ae55-cc42-431d-a27f-9417a6c581d0 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:03.601731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1345/2703] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:12.709871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:12.711303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:12.775561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc120e-43ce-4afa-bcaf-19844823f06a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:12.780666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:12.781388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:12.781697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:12.781979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] PATH_INFO: `/v3/users/e4346634525f46e3ba69da7ee990b786` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:12.783328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4346634525f46e3ba69da7ee990b786)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:12.785384 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:12.787191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:13.139758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4346634525f46e3ba69da7ee990b786'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:13.151775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4346634525f46e3ba69da7ee990b786'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:13.158773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4346634525f46e3ba69da7ee990b786'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:13.159506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4346634525f46e3ba69da7ee990b786'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:13.160120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4346634525f46e3ba69da7ee990b786'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:13.251212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1359/2704] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:25:12 2022] DELETE /identity/v3/users/e4346634525f46e3ba69da7ee990b786 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:13.256199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:13.256678 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:13.256805 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:13.277256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:13.278830 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:13.350969 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1346/2705] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:13.361314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:13.363319 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:13.419154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb41f6-92f3-44af-8a98-4db7182a5ee4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:13.423125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51262b5a-b45b-4bb1-bd88-8ab2a30931c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:13.423725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51262b5a-b45b-4bb1-bd88-8ab2a30931c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:13.424007 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51262b5a-b45b-4bb1-bd88-8ab2a30931c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:13.424270 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51262b5a-b45b-4bb1-bd88-8ab2a30931c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:13.433939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51262b5a-b45b-4bb1-bd88-8ab2a30931c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:13.435383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51262b5a-b45b-4bb1-bd88-8ab2a30931c5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:13.445352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1360/2706] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:13.879073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:13.880514 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:13.892981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf329f-7e4d-4fac-badf-fd224d0feeef None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:13.898146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:13.898877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:13.899153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:13.899419 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] PATH_INFO: `/v3/projects/860550889509456e867356246184afa5` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:13.900523 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=860550889509456e867356246184afa5)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:13.902425 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:13.919477 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:14.042205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860550889509456e867356246184afa5', 'request_id': 'req-921ce539-0e41-44b6-a5bb-5a8be1d986ff'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:14.131199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860550889509456e867356246184afa5', 'request_id': 'req-921ce539-0e41-44b6-a5bb-5a8be1d986ff'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:14.137142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860550889509456e867356246184afa5', 'request_id': 'req-921ce539-0e41-44b6-a5bb-5a8be1d986ff'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:14.148543 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] The token cache is being invalidate because project 860550889509456e867356246184afa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:25:14.148826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:14.151194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-921ce539-0e41-44b6-a5bb-5a8be1d986ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:14.151194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1347/2707] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:25:13 2022] DELETE /identity/v3/projects/860550889509456e867356246184afa5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:14.156249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.156767 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.157235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.178294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:14.179629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:14.245085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1361/2708] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:14.253093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.254012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.254176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.274087 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:14.275329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:14.283154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1348/2709] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:14.287732 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:14.297722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b14d66-6012-4615-9203-6c57f6aaa1a3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:14.303961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:14.304559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.304841 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.305115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.305788 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:25:14.306091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:14.307725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:14.320084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1362/2710] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:25:14.324689 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:14.335401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef7c9-4762-4b11-8371-920e0d88b928 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:14.340091 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:14.340749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.341093 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.341427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.342141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:14.344068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:14.462951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1349/2711] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:25:14 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:25:14.466978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:14.477669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b68a67-a2b2-4ccd-a358-af976b2f6197 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:14.483362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:14.538512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:14.539175 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.539450 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.539717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.540347 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:14.542236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:14.686004 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1363/2712] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:25:14 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:25:14.690188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:14.701204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a55b1-0484-4155-b34b-5baf3e62aceb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:14.705810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:14.706410 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.706704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.706979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.707613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:14.709211 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:14.719790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1350/2713] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:25:14.724057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:14.734282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ee508-38a2-40c3-95a7-75e3eaf04f83 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:14.739524 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f6d2a-6c45-412f-869a-b6cd6feded41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:14.740125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6d2a-6c45-412f-869a-b6cd6feded41 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.740406 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6d2a-6c45-412f-869a-b6cd6feded41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.740678 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6d2a-6c45-412f-869a-b6cd6feded41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.741323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f6d2a-6c45-412f-869a-b6cd6feded41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:14.742962 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f6d2a-6c45-412f-869a-b6cd6feded41 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:14.754478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1364/2714] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:25:14.758794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:14.769468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ea40a-9701-4d60-b35c-5917bf26befc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:14.774147 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:14.774835 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.775121 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.775389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] PATH_INFO: `/v3/projects/97f89fdfe9dd411cbf0433005f17f735/users/9c79ecdc2ce940f68dbf9d71a969e40b/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.790529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f89fdfe9dd411cbf0433005f17f735, user_id=9c79ecdc2ce940f68dbf9d71a969e40b, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:14.793193 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:14.914473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1351/2715] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:25:14 2022] PUT /identity/v3/projects/97f89fdfe9dd411cbf0433005f17f735/users/9c79ecdc2ce940f68dbf9d71a969e40b/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:14.926516 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977a2c-8c0b-4c5a-8d6b-5be7eb8c470d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:14.926830 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977a2c-8c0b-4c5a-8d6b-5be7eb8c470d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:14.927107 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977a2c-8c0b-4c5a-8d6b-5be7eb8c470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:14.957220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fb977a2c-8c0b-4c5a-8d6b-5be7eb8c470d None None] MFA Rules not processed for user `9c79ecdc2ce940f68dbf9d71a969e40b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:14.958439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fb977a2c-8c0b-4c5a-8d6b-5be7eb8c470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:15.039278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1365/2716] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:15.049637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:15.052372 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:15.121353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d00d-bdb0-4e42-af9c-bb75f0db1dfd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:15.125131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:15.125767 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:15.126387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:15.126387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:15.134912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:15.136330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:15.145056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1352/2717] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:24.544194 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:24.546228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:24.546228 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:24.571594 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] MFA Rules not processed for user `35af5d49420a437193f2525bda7550d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:24.572805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:24.631934 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1366/2718] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:24.705547 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:24.707705 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:24.758434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:24.774826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d48b5-84fa-4413-841c-c6c9c892852d service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:24.778511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:24.779152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:24.779430 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:24.779701 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:24.781894 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:24.793050 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:24.794414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:24.804272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1353/2719] 10.222.0.78 () {60 vars in 1396 bytes} [Wed Aug 10 23:25:24 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:25.044834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:25.054451 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7977b75-3877-40ef-9b66-55a780882ba5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:25.058065 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:25.058679 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:25.058969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:25.059238 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:25.061326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:25.072923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:25.074469 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:25.083314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1367/2720] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:25.515495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:25.517082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:25.590571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c7555-30ee-4538-b060-b03594cb000f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:25.595755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:25.596360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:25.597171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:25.597450 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:25.599209 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:25.610454 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:25.612129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e19ec-2cdd-4258-a3d2-76a9b124bb5e service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:25.622716 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1354/2721] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:29.149033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:29.158454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:29.215631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babaccc-bbbe-42eb-8232-32920084b0f4 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:29.221141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:29.221782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:29.222911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:29.222911 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] PATH_INFO: `/v3/users/9c79ecdc2ce940f68dbf9d71a969e40b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:29.224049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c79ecdc2ce940f68dbf9d71a969e40b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:29.225969 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:29.542274 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c79ecdc2ce940f68dbf9d71a969e40b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:29.563561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c79ecdc2ce940f68dbf9d71a969e40b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:29.570992 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c79ecdc2ce940f68dbf9d71a969e40b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:29.571728 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c79ecdc2ce940f68dbf9d71a969e40b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:29.572340 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c79ecdc2ce940f68dbf9d71a969e40b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:29.685514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1368/2722] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:25:29 2022] DELETE /identity/v3/users/9c79ecdc2ce940f68dbf9d71a969e40b => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:29.689884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bed84-506f-431d-ba64-9a8d603ce817 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:29.690395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bed84-506f-431d-ba64-9a8d603ce817 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:29.690619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bed84-506f-431d-ba64-9a8d603ce817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:29.711748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-414bed84-506f-431d-ba64-9a8d603ce817 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:29.712959 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-414bed84-506f-431d-ba64-9a8d603ce817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:29.772754 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1355/2723] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:29.782518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:29.783998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:29.842698 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f505358-8d07-4de9-8d46-ebe2804441a6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:29.846345 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:29.846982 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:29.847265 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:29.847538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:29.857752 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:29.859236 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:29.869733 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1369/2724] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:30.094082 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.094377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.094649 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.115960 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] MFA Rules not processed for user `05e18a4fd73e435aa6d0f10a37405698`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:30.117185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:30.193525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1356/2725] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:30.462556 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:30.464611 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:30.480756 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e517ddb-ef02-4fed-86a5-24f483f789af service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:30.484115 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:30.485099 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.485385 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.485663 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] PATH_INFO: `/v3/projects/97f89fdfe9dd411cbf0433005f17f735` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.486996 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f89fdfe9dd411cbf0433005f17f735)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:30.489836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:30.559068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:30.560495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:30.572185 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cdecd-2008-4bdc-a784-ab227d51b012 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:30.575633 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:30.576213 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.576488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.576755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.578378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:30.581182 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:30.588387 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:30.589814 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cc6cd-74d1-4cef-9b50-d89d6ab6050a service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:30.597782 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f89fdfe9dd411cbf0433005f17f735', 'request_id': 'req-31ff720c-a835-45d0-a305-372ed3def2b2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:30.598833 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1357/2726] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:30.685125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f89fdfe9dd411cbf0433005f17f735', 'request_id': 'req-31ff720c-a835-45d0-a305-372ed3def2b2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:30.691154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f89fdfe9dd411cbf0433005f17f735', 'request_id': 'req-31ff720c-a835-45d0-a305-372ed3def2b2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:30.703402 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] The token cache is being invalidate because project 97f89fdfe9dd411cbf0433005f17f735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:25:30.703769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:30.704494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31ff720c-a835-45d0-a305-372ed3def2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:25:30.706263 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1370/2727] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:25:30 2022] DELETE /identity/v3/projects/97f89fdfe9dd411cbf0433005f17f735 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:30.712153 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.712647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.714670 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.726537 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:30.732779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:30.733927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:30.793614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1358/2728] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:30.801872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.802158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.802432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.822209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:30.823453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:30.831614 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1371/2729] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:30.836813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:30.846233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a7d85-0ec4-4cea-ae58-646cd86255b9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:30.851220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:30.851714 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.855384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.855384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.855384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:25:30.855384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:30.855384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:30.866951 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1359/2730] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:25:30.871660 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:30.881253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcb339-4dc7-43dd-8748-22394bc20cb2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:30.885993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:30.886587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:30.886893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:30.887196 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:30.887854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:30.889819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:30.977597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1372/2731] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:25:30 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:25:30.981961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:30.992265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33fd47-6dd1-4067-b11c-576f082c0485 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:31.047531 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:31.048119 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:31.048394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:31.048663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:31.049266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:31.051711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:31.170899 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1360/2732] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:25:30 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:25:31.175183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:31.186109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3890309-2c49-4f47-95df-a13cdbf2ca3f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:31.190937 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:31.191535 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:31.192466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:31.192754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:31.193400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:31.195186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:31.205397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1373/2733] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:25:31.209902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:31.219797 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad642dc-da4a-4686-af4a-82742208f434 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:31.224443 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0322b46-3514-49f3-b957-f33132297c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:31.225034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0322b46-3514-49f3-b957-f33132297c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:31.225315 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0322b46-3514-49f3-b957-f33132297c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:31.225583 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0322b46-3514-49f3-b957-f33132297c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:31.226220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0322b46-3514-49f3-b957-f33132297c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:31.227824 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0322b46-3514-49f3-b957-f33132297c67 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:31.238497 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1361/2734] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:25:31.243478 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:31.253529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3a9d9-6fdb-4e21-a311-d5ba160f6846 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:31.258837 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:31.259481 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:31.259761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:31.260032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] PATH_INFO: `/v3/projects/1fbc9fef5e46490da9e1142d70e04228/users/ef47061e975849c6ab4054f380f8527c/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:31.275337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbc9fef5e46490da9e1142d70e04228, user_id=ef47061e975849c6ab4054f380f8527c, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:31.279379 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:31.405468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1374/2735] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:25:31 2022] PUT /identity/v3/projects/1fbc9fef5e46490da9e1142d70e04228/users/ef47061e975849c6ab4054f380f8527c/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:25:31.411042 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:31.411332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:31.411605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:31.432183 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:31.433522 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:31.556427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1362/2736] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:25:31.566235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:31.577229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:31.661294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba4ba-ae51-4aa6-a094-7e25c92df359 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:31.667442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ada4d-caf7-45e2-83cc-a57cc8677f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:31.670342 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ada4d-caf7-45e2-83cc-a57cc8677f06 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:31.670507 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ada4d-caf7-45e2-83cc-a57cc8677f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:31.670725 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ada4d-caf7-45e2-83cc-a57cc8677f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:31.686958 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ada4d-caf7-45e2-83cc-a57cc8677f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:31.688397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ada4d-caf7-45e2-83cc-a57cc8677f06 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:31.697995 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1375/2737] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:25:36.863285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:36.866133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:36.886803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:36.951381 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940de73-ed0c-4ea6-94d9-1c2a2b368fe5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:36.957656 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:36.958255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:36.958532 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:36.959276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:36.960829 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:36.980954 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:36.982712 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:36.995719 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1363/2738] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:38.252755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:38.252755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:38.252755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:38.282380 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] MFA Rules not processed for user `ef47061e975849c6ab4054f380f8527c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:38.283899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:38.333569 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:38.360489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1376/2739] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:38.371808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:38.394615 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fe0e4-d3cd-47a8-b80b-9a87e8b3470c service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:38.399282 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:38.399898 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:38.400168 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:38.400434 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:38.401048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:38.403036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:38.566282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:38.566988 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1364/2740] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:25:38.579047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb15ff-a596-423c-a878-b0caa23bd2d9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:38.649636 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:38.650146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:38.650494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:38.650880 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:38.651553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:38.653713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:38.762511 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1377/2741] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:25:38 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:25:38.764808 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:38.777013 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef014-97c4-495e-a46e-c9bda7887905 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:38.782575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:38.783472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:38.783749 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:38.784016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:38.784703 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:38.787103 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:38.806597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1365/2742] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:25:38.811538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:38.826304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520d99c-653d-4363-b5b8-6554aa04cb66 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:38.834710 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:38.835585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:38.836041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:38.836471 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] PATH_INFO: `/v3/projects/c0e869053ff848bfba779adcb43e6a37/users/fbfbc5309a5a4271a81f9a6ae0ffba1a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:38.862718 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e869053ff848bfba779adcb43e6a37, user_id=fbfbc5309a5a4271a81f9a6ae0ffba1a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:38.865448 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:38.970148 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1378/2743] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:25:38 2022] PUT /identity/v3/projects/c0e869053ff848bfba779adcb43e6a37/users/fbfbc5309a5a4271a81f9a6ae0ffba1a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:38.974264 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:38.985010 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae83a53-a605-4669-a373-46bdaae1491e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:39.051460 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fe7b2-dd55-4196-ae88-e65917a447c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:39.051981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fe7b2-dd55-4196-ae88-e65917a447c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:39.052278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fe7b2-dd55-4196-ae88-e65917a447c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:39.052520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fe7b2-dd55-4196-ae88-e65917a447c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:39.053156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fe7b2-dd55-4196-ae88-e65917a447c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:39.054806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fe7b2-dd55-4196-ae88-e65917a447c0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:39.065520 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1366/2744] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:25:39.070708 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:39.081737 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e5ae-915b-4d5a-9a75-ab65807d06da admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:39.089646 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:39.095191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:39.095191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:39.095191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] PATH_INFO: `/v3/projects/c0e869053ff848bfba779adcb43e6a37/users/fbfbc5309a5a4271a81f9a6ae0ffba1a/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:39.095191 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e869053ff848bfba779adcb43e6a37, user_id=fbfbc5309a5a4271a81f9a6ae0ffba1a, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:39.095643 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:39.107012 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1379/2745] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:25:39 2022] PUT /identity/v3/projects/c0e869053ff848bfba779adcb43e6a37/users/fbfbc5309a5a4271a81f9a6ae0ffba1a/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:25:45.785925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b0220-5350-4721-b0fb-055d3085d476 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:45.786394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b0220-5350-4721-b0fb-055d3085d476 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:45.786482 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b0220-5350-4721-b0fb-055d3085d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:45.815755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a26b0220-5350-4721-b0fb-055d3085d476 None None] MFA Rules not processed for user `fbfbc5309a5a4271a81f9a6ae0ffba1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:45.817000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a26b0220-5350-4721-b0fb-055d3085d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:45.867558 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:45.888155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1367/2746] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:25:45.900071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:45.901893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:45.985211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5bd3-ce66-4c22-b5da-2e43aff71150 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:45.989553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:45.990152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:45.990431 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:45.990770 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:46.021267 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:25:46.046491 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:46.047984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:46.057064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1380/2747] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:46.500216 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:46.500500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:46.500802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:46.556978 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:46.558225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:46.649416 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1368/2748] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:25:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:25:46.901827 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:46.906504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:47.011513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea33cc9-9885-40f9-b3d7-67aa1af7ac17 service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:47.017797 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:47.018489 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:47.018871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:47.019219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:47.027854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:47.029513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:47.038221 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1381/2749] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:25:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:25:54.545310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:54.612123 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6ccf-e755-46aa-aa2b-5087e835c47a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:54.617549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:54.618154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:54.618433 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:54.618727 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:54.620504 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:54.698219 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:54.699784 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7178242-1e11-422c-8bdc-524e18d14eac service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:54.714502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1369/2750] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:56.963707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:25:56.965495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:57.043110 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f0eca-939c-4a40-8dd3-c0f2268265df service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:25:57.049561 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:25:57.051025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:57.051025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:57.051335 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:57.053126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:57.065669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:25:57.067321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c335b2-bb72-40f4-8733-b6917dcf5430 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:25:57.078445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1382/2751] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:25:58.308927 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:25:58.309465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:25:58.309465 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:25:58.329293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] MFA Rules not processed for user `ef47061e975849c6ab4054f380f8527c`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:25:58.330475 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:25:58.337697 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1370/2752] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:25:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:26:01.824638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:26:01.825135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:26:01.825260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:26:01.845865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:26:01.847573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:26:01.854839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1383/2753] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 10 23:26:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:26:01.867210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:26:01.873329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:26:01.879626 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21524-05a4-4c12-840d-aba2a438458e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:26:01.885661 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:26:01.886386 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:26:01.886686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:26:01.889541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:26:01.896118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:26:01.897686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:26:01.905773 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1371/2754] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:26:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 23:26:11.839921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:26:12.080086 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:26:34.886513 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:26:34.895832 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce4cc6-66f9-42e2-a1b1-8ea2c27df352 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:26:34.899871 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:26:34.900444 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:26:34.900717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:26:34.900985 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:26:34.910064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:26:34.911515 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:26:34.921432 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1384/2755] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:26:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:26:44.917246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:26:47.592706 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:26:47.594126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:26:47.666125 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d9f8-d7d4-499f-9b20-6380bb10d700 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:26:47.672181 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:26:47.673589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:26:47.673589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:26:47.673589 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:26:47.675302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:26:47.687440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:26:47.689062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:26:47.699931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1372/2756] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:26:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:26:57.694905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=32 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:34.107894 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:34.116757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c478e2-ceb1-417f-a25d-07419abd10bf service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:34.120433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:34.121024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:34.121303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:34.121574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:34.123351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:34.185807 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:34.187273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:34.197332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1385/2757] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:38.010916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:38.010916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:38.024077 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faf853-e9bc-485b-a449-4bad4b3bc302 service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:38.028888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:38.029541 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:38.029843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:38.030116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] PATH_INFO: `/v3/users/35af5d49420a437193f2525bda7550d9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:38.031245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35af5d49420a437193f2525bda7550d9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:38.033130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:38.188604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35af5d49420a437193f2525bda7550d9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.201260 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35af5d49420a437193f2525bda7550d9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.208822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35af5d49420a437193f2525bda7550d9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.209577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35af5d49420a437193f2525bda7550d9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.210187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5b381db1-264c-4452-93ff-6601b8b3bc9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35af5d49420a437193f2525bda7550d9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.279569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1373/2758] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:27:38 2022] DELETE /identity/v3/users/35af5d49420a437193f2525bda7550d9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:38.581280 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:38.591512 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143657-3599-4027-bbbb-2e8d1b458fbd service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:38.650284 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:38.650940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:38.651220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:38.651494 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] PATH_INFO: `/v3/projects/e72afdc971a44dd6a4eaabd40c40bbcb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:38.652601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e72afdc971a44dd6a4eaabd40c40bbcb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:38.654562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:38.762864 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e72afdc971a44dd6a4eaabd40c40bbcb', 'request_id': 'req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.830751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e72afdc971a44dd6a4eaabd40c40bbcb', 'request_id': 'req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.836974 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e72afdc971a44dd6a4eaabd40c40bbcb', 'request_id': 'req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.848391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] The token cache is being invalidate because project e72afdc971a44dd6a4eaabd40c40bbcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:27:38.848677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.849303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ca00ae1-f240-49e4-aabf-3ea8eaa38320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:38.850785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1386/2759] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:27:38 2022] DELETE /identity/v3/projects/e72afdc971a44dd6a4eaabd40c40bbcb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:38.855777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:38.856068 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:38.856396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:38.876130 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:38.877331 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:38.937578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1374/2760] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:38.946352 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:38.946631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:38.946927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:38.966592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:38.967808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:38.976106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1387/2761] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:38.986021 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:38.997819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e99f7-033e-45f2-bc62-c48c8cf24955 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.003207 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.003798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.004072 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.004366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.004999 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:27:39.005290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.006878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.018968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1375/2762] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:27:39.023304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:39.033314 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0636-c334-415a-bc32-8594c248d363 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.038298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.038944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.039229 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.039502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.040140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.042034 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.058521 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:39.152328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1388/2763] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:27:39 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:39.156709 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:39.166613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7abcc0-bdd7-4151-a3bc-95db72d8332d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.221822 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.222402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.222693 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.222962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.223574 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.225468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.233224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:39.311033 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1376/2764] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:27:39 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:39.315017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:39.325436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f61d0d-1c19-49be-9264-3ce84c442e52 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.330044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.330635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.331233 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.331500 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.332180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.333755 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.343564 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1389/2765] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:39.348114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:39.358188 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6650d7-c4ec-4934-82d1-55e7704b51c7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.363370 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bd2eb-d61d-4075-ad89-95afdfe057f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.363956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bd2eb-d61d-4075-ad89-95afdfe057f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.364235 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bd2eb-d61d-4075-ad89-95afdfe057f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.364505 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bd2eb-d61d-4075-ad89-95afdfe057f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.365135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bd2eb-d61d-4075-ad89-95afdfe057f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.366765 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bd2eb-d61d-4075-ad89-95afdfe057f6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.377312 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1377/2766] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:39.381311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:39.391440 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec85f40-f36d-4176-b137-2ff4fece0548 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.396160 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.396802 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.397080 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.397354 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] PATH_INFO: `/v3/projects/04b15d4063dc465f8c5780f626f0477d/users/a00a44f349cb4d3f98ea26210d46a455/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.412241 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b15d4063dc465f8c5780f626f0477d, user_id=a00a44f349cb4d3f98ea26210d46a455, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.414952 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.505570 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1390/2767] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:27:39 2022] PUT /identity/v3/projects/04b15d4063dc465f8c5780f626f0477d/users/a00a44f349cb4d3f98ea26210d46a455/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:39.515803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.516090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.516367 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.545392 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] MFA Rules not processed for user `a00a44f349cb4d3f98ea26210d46a455`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:39.546601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:39.611715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1378/2768] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:39.623212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:39.624920 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:39.696784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13012a-3d80-4191-80a4-b6b0bdc630b3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:39.702206 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625159f-0050-40c0-84ef-3f3ca7f5a832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:39.702900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625159f-0050-40c0-84ef-3f3ca7f5a832 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:39.703177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625159f-0050-40c0-84ef-3f3ca7f5a832 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:39.703442 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625159f-0050-40c0-84ef-3f3ca7f5a832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:39.711631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625159f-0050-40c0-84ef-3f3ca7f5a832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:39.713179 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625159f-0050-40c0-84ef-3f3ca7f5a832 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:39.722477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1391/2769] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:48.909779 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:48.969303 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df0f8-494b-4829-aa82-903b93a08bbc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:48.973766 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:48.974411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:48.974717 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:48.974990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] PATH_INFO: `/v3/users/a00a44f349cb4d3f98ea26210d46a455` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:48.976127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00a44f349cb4d3f98ea26210d46a455)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:48.978031 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:49.181921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00a44f349cb4d3f98ea26210d46a455'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:49.194864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00a44f349cb4d3f98ea26210d46a455'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:49.199675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:49.202227 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00a44f349cb4d3f98ea26210d46a455'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:49.202957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00a44f349cb4d3f98ea26210d46a455'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:49.203567 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00a44f349cb4d3f98ea26210d46a455'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:49.270614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1379/2770] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:27:48 2022] DELETE /identity/v3/users/a00a44f349cb4d3f98ea26210d46a455 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:49.274893 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:49.275427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:49.275427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:49.295298 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:49.296520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:49.356669 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1392/2771] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:49.367326 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:49.368768 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:49.424753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df524a-c785-42f8-876b-1b226bfb58d5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:49.429251 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773d0af-a3c7-4a22-b1f0-ea0ed83be3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:49.429843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773d0af-a3c7-4a22-b1f0-ea0ed83be3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:49.430113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773d0af-a3c7-4a22-b1f0-ea0ed83be3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:49.430378 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773d0af-a3c7-4a22-b1f0-ea0ed83be3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:49.439442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773d0af-a3c7-4a22-b1f0-ea0ed83be3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:49.440860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773d0af-a3c7-4a22-b1f0-ea0ed83be3e8 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:49.450141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1380/2772] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:49.885957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:49.887423 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:49.900495 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cda85-074d-4058-b11a-dd28a7e8d59c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:49.905023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:49.905651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:49.905935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:49.906256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] PATH_INFO: `/v3/projects/04b15d4063dc465f8c5780f626f0477d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:49.907504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b15d4063dc465f8c5780f626f0477d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:49.909447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.052211 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:50.058674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b15d4063dc465f8c5780f626f0477d', 'request_id': 'req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:50.133896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b15d4063dc465f8c5780f626f0477d', 'request_id': 'req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:50.139993 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b15d4063dc465f8c5780f626f0477d', 'request_id': 'req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:50.152152 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] The token cache is being invalidate because project 04b15d4063dc465f8c5780f626f0477d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:27:50.152439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:50.153109 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd28c34a-4241-4bc9-ac22-d4f6316bae6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:50.154615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1393/2773] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:27:49 2022] DELETE /identity/v3/projects/04b15d4063dc465f8c5780f626f0477d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:50.159842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.160360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.160473 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.180132 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:50.181471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:50.243925 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1381/2774] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:50.253081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.253545 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.253822 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.273712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:50.275069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:50.283059 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1394/2775] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:50.287760 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:50.297524 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f828d59-7e52-43cf-bb49-866db4805ef8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:50.306411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:50.307053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.307334 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.307604 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.308249 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:27:50.308549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:50.310149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.322508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1382/2776] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:27:50.326899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:50.336593 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8675d5-7bf8-47e3-905a-dfb861e654c0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:50.341668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:50.341891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.342209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.342548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.343266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:50.345533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.467712 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1395/2777] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:27:50 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:50.468782 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:50.479955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b5786-9eea-444e-b0a1-db0c90f96ede admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:50.539002 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:50.539721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.540003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.541286 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.541917 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:50.543852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.637148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1383/2778] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:27:50 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:50.641453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:50.652703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d4294-26ef-4274-957a-1c5b65d224d3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:50.657328 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:50.657936 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.658214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.658484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.659157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:50.662951 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.671775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1396/2779] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:50.676008 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:50.688673 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec9627-0d48-401a-a54c-a4e2908b4cd6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:50.697041 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91109d7d-971b-44da-86d5-69956fc6054d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:50.697648 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91109d7d-971b-44da-86d5-69956fc6054d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.697918 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91109d7d-971b-44da-86d5-69956fc6054d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.698187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91109d7d-971b-44da-86d5-69956fc6054d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.698885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91109d7d-971b-44da-86d5-69956fc6054d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:50.702802 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91109d7d-971b-44da-86d5-69956fc6054d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.704607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:50.721242 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1384/2780] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:27:50.725327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:50.735368 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2fc0-ec60-4952-b551-e7af571ec351 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:50.739941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:50.740585 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.740866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.741138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] PATH_INFO: `/v3/projects/48919ea9e0cb4c6c9752e02901ee304f/users/bc69dd31e08c4b22a6bcd9584c9c1ee4/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.756475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48919ea9e0cb4c6c9752e02901ee304f, user_id=bc69dd31e08c4b22a6bcd9584c9c1ee4, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:50.759219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:50.839941 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1397/2781] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:27:50 2022] PUT /identity/v3/projects/48919ea9e0cb4c6c9752e02901ee304f/users/bc69dd31e08c4b22a6bcd9584c9c1ee4/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:50.850316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:50.850601 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:50.850901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:50.877401 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] MFA Rules not processed for user `bc69dd31e08c4b22a6bcd9584c9c1ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:50.878620 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:50.941090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1385/2782] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:50.946158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.007047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e47f5-c4d3-4ae5-a381-df35f3ee1689 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.013321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.013919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.014197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.014468 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.015118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.017001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.120504 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1398/2783] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:27:50 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:51.124538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.135486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e50884-4a9a-461b-a08e-e5fd804665de None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.190194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.190907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.191112 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.191351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.192012 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.194556 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.287710 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1386/2784] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:27:51 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:51.291799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.302895 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41ce7-409e-48b6-87e6-fd4d49a1583e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.307454 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.308041 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.308312 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.308578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.309257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.310865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.323751 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1399/2785] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:51.327982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.338389 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dab163-21d5-48d6-bbff-3c514306e42a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.342922 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.343555 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.343832 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.344104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] PATH_INFO: `/v3/projects/e0fde20bddd447608dd801106e36bb0a/users/50ad9068b6db4d688f2f2b9a9ab19b8d/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.359359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fde20bddd447608dd801106e36bb0a, user_id=50ad9068b6db4d688f2f2b9a9ab19b8d, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.361961 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.448526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1387/2786] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:27:51 2022] PUT /identity/v3/projects/e0fde20bddd447608dd801106e36bb0a/users/50ad9068b6db4d688f2f2b9a9ab19b8d/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:51.452554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.463283 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260ee3e-f898-4d27-93b1-8a7e2b2016ce admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.499997 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:51.518389 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2331315-94c6-4d0f-b926-fa2b07c6dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.519011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2331315-94c6-4d0f-b926-fa2b07c6dcce admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.519287 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2331315-94c6-4d0f-b926-fa2b07c6dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.519553 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2331315-94c6-4d0f-b926-fa2b07c6dcce admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.520177 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2331315-94c6-4d0f-b926-fa2b07c6dcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.521790 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2331315-94c6-4d0f-b926-fa2b07c6dcce admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.532204 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1400/2787] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:27:51.536269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.547432 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f45a0f-2f41-4234-b6cc-9270d7a10ad2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.551926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.553127 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.553399 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.553665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] PATH_INFO: `/v3/projects/e0fde20bddd447608dd801106e36bb0a/users/50ad9068b6db4d688f2f2b9a9ab19b8d/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.555664 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fde20bddd447608dd801106e36bb0a, user_id=50ad9068b6db4d688f2f2b9a9ab19b8d, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.558308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.568310 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1388/2788] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:27:51 2022] PUT /identity/v3/projects/e0fde20bddd447608dd801106e36bb0a/users/50ad9068b6db4d688f2f2b9a9ab19b8d/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:51.578565 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.578875 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.579141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.607701 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] MFA Rules not processed for user `50ad9068b6db4d688f2f2b9a9ab19b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:51.609172 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:51.679503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1401/2789] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:51.691150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.692871 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:51.736571 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:51.752607 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec6f47-34cc-46ff-8ac5-92cb5a357eee admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.756496 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241c870-0dc7-426e-afff-96112b61c784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.757089 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c870-0dc7-426e-afff-96112b61c784 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.757365 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c870-0dc7-426e-afff-96112b61c784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.757698 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c870-0dc7-426e-afff-96112b61c784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.767512 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241c870-0dc7-426e-afff-96112b61c784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.769051 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241c870-0dc7-426e-afff-96112b61c784 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:51.779911 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1389/2790] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:51.848030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:51.909341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1eb9d8-3545-48ef-8999-c371c81cf14a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:51.914498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:51.915159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:51.915439 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:51.915873 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] PATH_INFO: `/v3/users/bc69dd31e08c4b22a6bcd9584c9c1ee4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:51.916998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc69dd31e08c4b22a6bcd9584c9c1ee4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:51.918926 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:52.165859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc69dd31e08c4b22a6bcd9584c9c1ee4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:52.176706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:52.177810 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc69dd31e08c4b22a6bcd9584c9c1ee4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:52.184460 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc69dd31e08c4b22a6bcd9584c9c1ee4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:52.185143 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc69dd31e08c4b22a6bcd9584c9c1ee4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:52.185734 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc69dd31e08c4b22a6bcd9584c9c1ee4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:52.288242 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1402/2791] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:27:51 2022] DELETE /identity/v3/users/bc69dd31e08c4b22a6bcd9584c9c1ee4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:52.292874 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:52.293359 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:52.293463 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:52.312206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:52.313408 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:52.371888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1390/2792] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:52.382606 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:52.384186 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:52.444422 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92515883-4793-4899-b0ed-2958dddf747d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:52.448139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:52.448736 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:52.449018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:52.449286 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:52.459474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:52.460961 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:52.470470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1403/2793] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:52.903511 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:52.905020 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:52.918967 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3582c-04bf-43af-bab5-576b14d123ad None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:52.923857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:52.924777 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:52.924939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:52.925278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] PATH_INFO: `/v3/projects/48919ea9e0cb4c6c9752e02901ee304f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:52.926474 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48919ea9e0cb4c6c9752e02901ee304f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:52.929022 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:53.062495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48919ea9e0cb4c6c9752e02901ee304f', 'request_id': 'req-3c75bd3f-140e-4213-a2f4-c59229a2abe5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.141915 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48919ea9e0cb4c6c9752e02901ee304f', 'request_id': 'req-3c75bd3f-140e-4213-a2f4-c59229a2abe5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.148979 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48919ea9e0cb4c6c9752e02901ee304f', 'request_id': 'req-3c75bd3f-140e-4213-a2f4-c59229a2abe5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.160225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] The token cache is being invalidate because project 48919ea9e0cb4c6c9752e02901ee304f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:27:53.160508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.161128 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.162579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1391/2794] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:27:52 2022] DELETE /identity/v3/projects/48919ea9e0cb4c6c9752e02901ee304f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:53.166617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:53.176257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090a3e-96ae-42cf-b8c8-874e863f5c53 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:53.231562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:53.232100 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:53.232372 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:53.232638 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] PATH_INFO: `/v3/users/50ad9068b6db4d688f2f2b9a9ab19b8d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:53.233779 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ad9068b6db4d688f2f2b9a9ab19b8d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:53.235657 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:53.416151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ad9068b6db4d688f2f2b9a9ab19b8d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.435091 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ad9068b6db4d688f2f2b9a9ab19b8d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.442427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ad9068b6db4d688f2f2b9a9ab19b8d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.443223 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ad9068b6db4d688f2f2b9a9ab19b8d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.443829 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cd92cbbf-b511-4770-aa27-5c07df9ee515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ad9068b6db4d688f2f2b9a9ab19b8d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:53.540081 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1404/2795] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:27:53 2022] DELETE /identity/v3/users/50ad9068b6db4d688f2f2b9a9ab19b8d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:53.998075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:53.999563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:54.015737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75bd3f-140e-4213-a2f4-c59229a2abe5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:54.076820 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:54.077330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.077606 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.077876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] PATH_INFO: `/v3/projects/e0fde20bddd447608dd801106e36bb0a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.079024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fde20bddd447608dd801106e36bb0a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:54.081395 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:54.088973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:54.268278 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fde20bddd447608dd801106e36bb0a', 'request_id': 'req-c7dc4694-d271-460c-8802-bfc4feb5a9a4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:54.431247 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fde20bddd447608dd801106e36bb0a', 'request_id': 'req-c7dc4694-d271-460c-8802-bfc4feb5a9a4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:54.438841 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fde20bddd447608dd801106e36bb0a', 'request_id': 'req-c7dc4694-d271-460c-8802-bfc4feb5a9a4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:54.450732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] The token cache is being invalidate because project e0fde20bddd447608dd801106e36bb0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:27:54.450996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:54.453735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c7dc4694-d271-460c-8802-bfc4feb5a9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:54.453735 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1392/2796] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:27:53 2022] DELETE /identity/v3/projects/e0fde20bddd447608dd801106e36bb0a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:54.459155 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.459697 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.460252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.469288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:54.481060 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:54.482249 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:54.541703 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1405/2797] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:54.550684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.550981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.551287 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.571255 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:54.576285 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:54.580741 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1393/2798] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:54.585015 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:54.595047 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb59c5-8b61-4f97-b5b3-57fdce5c2604 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:54.601125 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:54.601874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.602055 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.602323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.603032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:27:54.603333 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:54.604922 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:54.616877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1406/2799] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:27:54.621196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:54.630444 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22546d-316d-4008-b09d-6b07f9e617c1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:54.635269 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:54.635877 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.636150 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.636415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.637027 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:54.638939 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:54.740826 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1394/2800] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:27:54 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:54.745031 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:54.754674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6ff7c-3721-4d4f-b2d3-e3de674db0c2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:54.808559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:54.809159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.809436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.809706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.810323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:54.812268 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:54.916950 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1407/2801] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:27:54 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:54.921220 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:54.932241 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42460648-f114-43a9-ac30-caa4691db670 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:54.936946 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:54.937526 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.937799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.938067 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.938753 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:54.940314 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:54.950075 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1395/2802] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:54.954070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:54.965262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d3ac5-8e18-4074-99c9-1b25cc4a9acc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:54.969877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49725e2c-ae86-4065-9055-e7951d8240d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:54.973652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725e2c-ae86-4065-9055-e7951d8240d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:54.973652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725e2c-ae86-4065-9055-e7951d8240d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:54.973652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725e2c-ae86-4065-9055-e7951d8240d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:54.973652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49725e2c-ae86-4065-9055-e7951d8240d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:54.973652 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49725e2c-ae86-4065-9055-e7951d8240d3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:54.984273 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1408/2803] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:54.988380 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:54.998447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048e7d-d2f9-4d62-9169-4c08f594cc5a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:55.002982 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:55.003612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:55.003885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:55.004154 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] PATH_INFO: `/v3/projects/46a3698766fa49dbb9ad23c4958e631e/users/3df83d9fcfd9410da4c9d22999dae7ec/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:55.018292 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a3698766fa49dbb9ad23c4958e631e, user_id=3df83d9fcfd9410da4c9d22999dae7ec, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:55.021487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:55.107550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1396/2804] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:27:54 2022] PUT /identity/v3/projects/46a3698766fa49dbb9ad23c4958e631e/users/3df83d9fcfd9410da4c9d22999dae7ec/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:55.118032 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62c47-b444-4115-851f-d41b25004186 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:55.118349 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62c47-b444-4115-851f-d41b25004186 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:55.118620 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62c47-b444-4115-851f-d41b25004186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:55.152554 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-87d62c47-b444-4115-851f-d41b25004186 None None] MFA Rules not processed for user `3df83d9fcfd9410da4c9d22999dae7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:55.153783 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87d62c47-b444-4115-851f-d41b25004186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:55.219876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:55.229262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1409/2805] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:27:55.237288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:55.239036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:55.297867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6265be9-59f7-4b68-921a-d1370c994c67 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:55.301551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fb3de-c04c-490d-acf1-abfa44528fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:55.302152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fb3de-c04c-490d-acf1-abfa44528fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:55.302431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fb3de-c04c-490d-acf1-abfa44528fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:55.302732 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fb3de-c04c-490d-acf1-abfa44528fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:55.311565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fb3de-c04c-490d-acf1-abfa44528fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:55.313040 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fb3de-c04c-490d-acf1-abfa44528fb4 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:55.322681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1397/2806] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:55.344589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62c47-b444-4115-851f-d41b25004186 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:55.406316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62c47-b444-4115-851f-d41b25004186 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:55.410892 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:55.411529 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:55.411808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:55.412076 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] PATH_INFO: `/v3/users/3df83d9fcfd9410da4c9d22999dae7ec` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:55.413192 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df83d9fcfd9410da4c9d22999dae7ec)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:55.415121 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:55.673173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df83d9fcfd9410da4c9d22999dae7ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:55.685900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df83d9fcfd9410da4c9d22999dae7ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:55.692651 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df83d9fcfd9410da4c9d22999dae7ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:55.693326 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df83d9fcfd9410da4c9d22999dae7ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:55.693927 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df83d9fcfd9410da4c9d22999dae7ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:55.808021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1410/2807] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:27:55 2022] DELETE /identity/v3/users/3df83d9fcfd9410da4c9d22999dae7ec => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:55.812456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:55.812748 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:55.813049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:55.832737 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:55.834441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:55.862288 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:55.891495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1398/2808] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:55.901662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:55.903711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:55.960101 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522ec4a-33f3-4270-82c7-3f915cb22010 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:55.963676 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc1b81-02d0-4a5f-9eb4-0eb6896a277f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:55.964266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc1b81-02d0-4a5f-9eb4-0eb6896a277f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:55.964542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc1b81-02d0-4a5f-9eb4-0eb6896a277f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:55.964809 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc1b81-02d0-4a5f-9eb4-0eb6896a277f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:55.973670 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc1b81-02d0-4a5f-9eb4-0eb6896a277f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:55.975717 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc1b81-02d0-4a5f-9eb4-0eb6896a277f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:55.984977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1411/2809] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:27:56.438949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:56.450092 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:56.455840 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a3ee8-fbd5-47c6-a0b3-5b6a68fc73cd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:56.465073 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:56.465843 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:56.465985 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:56.466394 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] PATH_INFO: `/v3/projects/46a3698766fa49dbb9ad23c4958e631e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:56.467645 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a3698766fa49dbb9ad23c4958e631e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:56.469816 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:56.613546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a3698766fa49dbb9ad23c4958e631e', 'request_id': 'req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:56.704254 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a3698766fa49dbb9ad23c4958e631e', 'request_id': 'req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:56.710299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a3698766fa49dbb9ad23c4958e631e', 'request_id': 'req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:56.721245 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] The token cache is being invalidate because project 46a3698766fa49dbb9ad23c4958e631e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:27:56.721466 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:56.722094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b8614e2e-fb09-455d-9ce0-3ab62ec9a994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:27:56.723586 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1399/2810] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:27:56 2022] DELETE /identity/v3/projects/46a3698766fa49dbb9ad23c4958e631e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:56.728487 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:56.729017 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:56.729135 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:56.748470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:56.749674 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:56.762877 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:56.814677 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1412/2811] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:56.822575 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:56.822878 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:56.823149 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:56.842952 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:56.844290 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:56.852611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1400/2812] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:56.856884 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:56.867209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68c46c-1253-48bb-8fd0-4c66a6509c5d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:56.873336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:56.873336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:56.873336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:56.873336 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:56.873754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:27:56.873754 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:56.875348 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:56.886573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1413/2813] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:27:56.890990 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:56.900142 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207f5d-fa4f-4067-bb75-d0493ad1d363 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:56.904681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:56.905256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:56.905525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:56.905787 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:56.906402 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:56.908308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:57.020194 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1401/2814] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:27:56 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:57.024477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:57.034456 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc53a6-ac9a-4625-9d1c-a53907797215 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:57.091978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:57.092575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:57.092850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:57.093118 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:57.093735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:57.095720 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:57.248183 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1414/2815] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:27:57 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:27:57.252305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:57.262467 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd122e5-be10-4ce4-86df-193e5debcb51 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:57.266926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:57.267507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:57.267781 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:57.268049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:57.268674 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:57.270317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:57.280611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1402/2816] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:57.284707 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:57.295300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af236c40-f40b-4045-a56a-b5ddee1470a6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:57.299773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bf8f8-4cda-4e79-8f28-74689708a9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:57.300359 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bf8f8-4cda-4e79-8f28-74689708a9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:57.300631 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bf8f8-4cda-4e79-8f28-74689708a9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:57.300899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bf8f8-4cda-4e79-8f28-74689708a9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:57.301518 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bf8f8-4cda-4e79-8f28-74689708a9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:57.303128 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bf8f8-4cda-4e79-8f28-74689708a9b6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:57.313575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1415/2817] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:27:57.317854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:57.327888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe53c8-beca-4dbe-99c0-98aa6b7edb44 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:57.332715 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:57.333851 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:57.334129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:57.334404 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] PATH_INFO: `/v3/projects/2dd2d6bd98204a71bbeda00c5e7141cd/users/7074b099a416466bb3c2e40f0bafa65b/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:57.348451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd2d6bd98204a71bbeda00c5e7141cd, user_id=7074b099a416466bb3c2e40f0bafa65b, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:57.351755 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:57.490643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1403/2818] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:27:57 2022] PUT /identity/v3/projects/2dd2d6bd98204a71bbeda00c5e7141cd/users/7074b099a416466bb3c2e40f0bafa65b/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:27:57.557113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f931d5-b41e-4d03-904a-382b0690725a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:57.557316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f931d5-b41e-4d03-904a-382b0690725a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:57.557600 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f931d5-b41e-4d03-904a-382b0690725a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:57.588225 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d9f931d5-b41e-4d03-904a-382b0690725a None None] MFA Rules not processed for user `7074b099a416466bb3c2e40f0bafa65b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:27:57.589449 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d9f931d5-b41e-4d03-904a-382b0690725a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:57.656214 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1416/2819] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:27:57.668187 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:27:57.669926 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:27:57.730828 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f953a-8a96-4504-94f5-b538e456010d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:27:57.734495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28dc00f9c744cc4b103b8e01d65f75f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:27:57.735124 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:27:57.735397 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:27:57.735663 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:27:57.747488 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:27:57.750210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:27:57.751913 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:27:57.762214 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1404/2820] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:27:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:07.653220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:07.918362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:18.733988 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a054fb-bb61-40c1-be05-4357b9c83b42 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:18.734477 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a054fb-bb61-40c1-be05-4357b9c83b42 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:18.735049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a054fb-bb61-40c1-be05-4357b9c83b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:18.755153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-40a054fb-bb61-40c1-be05-4357b9c83b42 None None] MFA Rules not processed for user `05e18a4fd73e435aa6d0f10a37405698`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:18.756375 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-40a054fb-bb61-40c1-be05-4357b9c83b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:18.813984 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1417/2821] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:22.527468 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:22.588872 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b549-49f7-46f3-93b7-5886f6ada730 service glance] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:22.593860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:22.594487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:22.594791 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:22.595063 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] PATH_INFO: `/v3/users/7074b099a416466bb3c2e40f0bafa65b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:22.596175 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7074b099a416466bb3c2e40f0bafa65b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:22.598106 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:22.852113 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7074b099a416466bb3c2e40f0bafa65b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:22.864900 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7074b099a416466bb3c2e40f0bafa65b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:22.871632 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7074b099a416466bb3c2e40f0bafa65b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:22.872345 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7074b099a416466bb3c2e40f0bafa65b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:22.872944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7074b099a416466bb3c2e40f0bafa65b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:22.992192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1405/2822] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:28:22 2022] DELETE /identity/v3/users/7074b099a416466bb3c2e40f0bafa65b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:28:22.996917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:22.997427 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:22.997562 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:23.018262 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:23.019505 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:23.077400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1418/2823] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:23.087492 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:23.088919 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:23.147947 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279d827-c290-4ab6-a02b-da6f059ac21f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:23.151933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24cfa-06c3-4e3d-8233-fd95e6583a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:23.152534 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24cfa-06c3-4e3d-8233-fd95e6583a63 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:23.152842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24cfa-06c3-4e3d-8233-fd95e6583a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:23.153681 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24cfa-06c3-4e3d-8233-fd95e6583a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:23.163759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d24cfa-06c3-4e3d-8233-fd95e6583a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:23.165548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d24cfa-06c3-4e3d-8233-fd95e6583a63 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:23.178136 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1406/2824] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:23.583141 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:23.584557 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:23.597027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbab702-fd35-458a-b1d6-e987ca362eaf None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:23.602337 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:23.602533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:23.602842 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:23.603113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] PATH_INFO: `/v3/projects/2dd2d6bd98204a71bbeda00c5e7141cd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:23.604235 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd2d6bd98204a71bbeda00c5e7141cd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:23.606686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:23.757171 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd2d6bd98204a71bbeda00c5e7141cd', 'request_id': 'req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:23.818219 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd2d6bd98204a71bbeda00c5e7141cd', 'request_id': 'req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:23.824085 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd2d6bd98204a71bbeda00c5e7141cd', 'request_id': 'req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:23.835847 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] The token cache is being invalidate because project 2dd2d6bd98204a71bbeda00c5e7141cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:28:23.836069 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:23.836548 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0e14a7b2-d23f-42a2-8dad-63a90d9d05f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:23.838013 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1419/2825] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:28:23 2022] DELETE /identity/v3/projects/2dd2d6bd98204a71bbeda00c5e7141cd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:23.843056 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:23.843549 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:23.843659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:23.863801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:23.865006 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:23.882414 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:23.923598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1407/2826] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:23.931316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:23.931729 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:23.932033 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:23.951846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:23.953056 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:23.961455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1420/2827] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:28:23.966308 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:23.976382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7618c-d920-4b09-8894-141f093b6884 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:23.980923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:23.982061 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:23.982332 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:23.982598 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:23.983256 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:28:23.983551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:23.985109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:23.996366 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1408/2828] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:28:24.000899 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:24.010112 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02787fe1-a3d1-4896-9852-f4648965dafc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:24.014702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:24.015294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.015568 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.015836 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.016457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:24.018356 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:24.128232 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:24.134546 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1421/2829] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:28:23 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:28:24.138803 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:24.148651 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c84b9f-16d1-4494-8185-49d67c4c9f12 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:24.207360 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:24.207996 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.208272 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.208544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.209156 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:24.211100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:24.325016 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1409/2830] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:28:24 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:28:24.329181 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:24.339940 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530da4c-4d36-48e7-b99d-daa4b74bd96c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:24.345346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:24.346166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.346502 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.346799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.347424 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:24.348998 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:24.360777 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1422/2831] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:28:24.364776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:24.374837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de081d-1eb1-4847-95a6-92cbf991aff3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:24.379398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f396f1-d375-4d28-9af9-73ead61e42b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:24.380000 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f396f1-d375-4d28-9af9-73ead61e42b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.380275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f396f1-d375-4d28-9af9-73ead61e42b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.380539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f396f1-d375-4d28-9af9-73ead61e42b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.381173 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f396f1-d375-4d28-9af9-73ead61e42b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:24.383957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f396f1-d375-4d28-9af9-73ead61e42b9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:24.395007 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1410/2832] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:28:24.399261 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:24.409706 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d3b9e-eabb-4fdc-b2d7-d226d1239b12 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:24.414157 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:24.415165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.415309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.415467 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] PATH_INFO: `/v3/projects/0d752360d5ea4850abe41127d772bb65/users/9962ee8950124c418bc453930b72f7f2/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.433291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d752360d5ea4850abe41127d772bb65, user_id=9962ee8950124c418bc453930b72f7f2, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:24.435999 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:24.588800 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1423/2833] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:28:24 2022] PUT /identity/v3/projects/0d752360d5ea4850abe41127d772bb65/users/9962ee8950124c418bc453930b72f7f2/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:24.609683 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.610271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.610772 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.668529 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] MFA Rules not processed for user `9962ee8950124c418bc453930b72f7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:24.670484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:24.740551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1411/2834] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:24.752851 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:24.754637 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:24.812376 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054789-f743-4da6-b519-b007814654e0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:24.816021 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:24.816617 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:24.816900 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:24.817169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:24.826475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:24.827921 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:24.837253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1424/2835] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:33.545199 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:33.546931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:33.567192 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:33.602597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c935916-f887-4453-b284-fd4fae32ad87 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:33.606036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:33.607171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:33.607445 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:33.607726 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:33.609396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:33.680052 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:33.681500 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:33.693403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1412/2836] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:34.833291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:34.871555 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:34.881408 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2e258-0fc4-4448-b286-f4ed45972e2d service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:34.884972 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:34.885542 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:34.885818 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:34.886084 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:34.887799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:34.901793 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:34.903230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7df00e-ff4a-42ed-9caf-8f8e688e1346 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:34.914250 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1425/2837] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:43.772699 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=46 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:44.938377 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:52.251895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:52.314398 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0322-f3b8-48ec-868a-3d6cb4aa44da service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:52.319964 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:52.320590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:52.320867 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:52.321138 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] PATH_INFO: `/v3/users/9962ee8950124c418bc453930b72f7f2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:52.322250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9962ee8950124c418bc453930b72f7f2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:52.324204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:52.504818 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9962ee8950124c418bc453930b72f7f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:52.520126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9962ee8950124c418bc453930b72f7f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:52.528224 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9962ee8950124c418bc453930b72f7f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:52.529116 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9962ee8950124c418bc453930b72f7f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:52.529817 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9962ee8950124c418bc453930b72f7f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:52.598481 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1413/2838] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:28:52 2022] DELETE /identity/v3/users/9962ee8950124c418bc453930b72f7f2 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:52.603302 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:52.603804 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:52.603918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:52.624362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:52.625497 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:52.690227 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1426/2839] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:52.699860 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:52.701383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:52.771440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db17fc9-0ac1-41bd-b95a-6ddf44e8cb38 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:52.775210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdafdb-be81-47f3-af40-0b9bfb6a9707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:52.775815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdafdb-be81-47f3-af40-0b9bfb6a9707 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:52.776094 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdafdb-be81-47f3-af40-0b9bfb6a9707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:52.776407 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdafdb-be81-47f3-af40-0b9bfb6a9707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:52.786975 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdafdb-be81-47f3-af40-0b9bfb6a9707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:52.788640 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdafdb-be81-47f3-af40-0b9bfb6a9707 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:52.798901 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1414/2840] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:53.244863 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:53.246544 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:53.260655 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36968b-e2cc-493f-827c-359f1fc62014 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:53.266474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:53.267285 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.268209 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.268582 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] PATH_INFO: `/v3/projects/0d752360d5ea4850abe41127d772bb65` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.270120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d752360d5ea4850abe41127d772bb65)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:53.272437 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:53.394142 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d752360d5ea4850abe41127d772bb65', 'request_id': 'req-25864d3d-9268-4093-8553-02bdcc6252a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:53.463566 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d752360d5ea4850abe41127d772bb65', 'request_id': 'req-25864d3d-9268-4093-8553-02bdcc6252a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:53.469867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d752360d5ea4850abe41127d772bb65', 'request_id': 'req-25864d3d-9268-4093-8553-02bdcc6252a3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:53.481332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] The token cache is being invalidate because project 0d752360d5ea4850abe41127d772bb65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:28:53.481719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:53.482436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-25864d3d-9268-4093-8553-02bdcc6252a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:28:53.484022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1427/2841] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:28:53 2022] DELETE /identity/v3/projects/0d752360d5ea4850abe41127d772bb65 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:53.489643 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.490151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.490151 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.510029 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:53.511266 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:53.527941 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:53.571248 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1415/2842] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:53.579510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.584050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.584050 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.599578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:53.600792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:53.609719 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1428/2843] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:53.614994 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:53.624569 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e313dd-c338-46c4-b49d-cc9e74ad5806 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:53.630117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:53.630984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.631764 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.632206 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.632846 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:28:53.633157 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:53.634756 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:53.646305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1416/2844] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:28:53.650799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:53.660579 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a271d6-cd99-442f-a94c-f9aeb8f1937b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:53.665786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:53.670104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.670104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.670104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.670104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:53.670104 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:53.757339 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1429/2845] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:28:53 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:28:53.758675 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:53.768905 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff8ed-c408-4cc4-947c-48e2f364ecc7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:53.832665 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:53.833431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.833800 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.834122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.834837 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:53.836940 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:53.921265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1417/2846] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:28:53 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:28:53.926113 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:53.936761 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701cfd2-0e83-44ee-9d44-ff31a964cb5a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:53.941484 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:53.942252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.942633 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.942943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.943551 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:53.945146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:53.955702 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1430/2847] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:28:53.959870 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:53.970205 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb59a9-34fb-4551-b758-6a3160c88c04 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:53.975902 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c8ffb-b172-498d-8aa6-cd7e10f8f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:53.976495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c8ffb-b172-498d-8aa6-cd7e10f8f088 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:53.976776 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c8ffb-b172-498d-8aa6-cd7e10f8f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:53.977046 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c8ffb-b172-498d-8aa6-cd7e10f8f088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:53.977684 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c8ffb-b172-498d-8aa6-cd7e10f8f088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:53.979302 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c8ffb-b172-498d-8aa6-cd7e10f8f088 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:53.992317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1418/2848] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:28:53.996383 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.006865 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4c6f6-3047-4a43-a6d4-27826fe62745 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.011615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.012180 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.012455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.012723 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] PATH_INFO: `/v3/projects/205e6728f1d04fcebaa70f337be35775/users/b22ad986d925481e8e5fe661eea0a783/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.027866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205e6728f1d04fcebaa70f337be35775, user_id=b22ad986d925481e8e5fe661eea0a783, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.030509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:54.118721 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1431/2849] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:28:53 2022] PUT /identity/v3/projects/205e6728f1d04fcebaa70f337be35775/users/b22ad986d925481e8e5fe661eea0a783/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:54.131383 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.131659 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.131921 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.160591 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] MFA Rules not processed for user `b22ad986d925481e8e5fe661eea0a783`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:54.161992 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:54.228734 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1419/2850] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:54.238979 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.259479 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:54.303266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c487-bbfa-4222-bd6d-dce31a252d49 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.308601 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.309213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.309509 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.309773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.310398 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.313300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:54.386278 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1432/2851] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:28:54 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:28:54.390166 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.400955 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a686cb8-2ebc-4b28-a6c0-08fe8fd12917 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.413015 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:54.464559 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.465440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.465956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.466641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.467476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.469441 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:54.582687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1420/2852] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:28:54 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:28:54.585173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.598039 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb9778-23e4-40b4-885f-cc1b166b763d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.603784 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.604388 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.604688 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.604957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.605588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.607257 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:54.619291 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1433/2853] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:28:54.623105 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.634148 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69708d00-83f0-4934-9be0-712824b48520 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.639597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f199-7ba2-4fb7-8fa9-1e29e7a4629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.640842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f199-7ba2-4fb7-8fa9-1e29e7a4629e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.640842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f199-7ba2-4fb7-8fa9-1e29e7a4629e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.640842 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f199-7ba2-4fb7-8fa9-1e29e7a4629e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.641368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f199-7ba2-4fb7-8fa9-1e29e7a4629e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.643019 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f199-7ba2-4fb7-8fa9-1e29e7a4629e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:54.653479 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1421/2854] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:28:54.657527 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.668247 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e86d3-d9d3-4eac-93e8-16c739b330b0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.672953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.673599 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.674726 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.675006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] PATH_INFO: `/v3/projects/e9173d501b88458ca40dce2960901f18/users/5ad62d7441ba4942848cc34a235ff619/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.691026 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9173d501b88458ca40dce2960901f18, user_id=5ad62d7441ba4942848cc34a235ff619, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.693700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:54.757680 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1434/2855] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:28:54 2022] PUT /identity/v3/projects/e9173d501b88458ca40dce2960901f18/users/5ad62d7441ba4942848cc34a235ff619/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:54.767815 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.768133 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.768472 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.797171 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] MFA Rules not processed for user `5ad62d7441ba4942848cc34a235ff619`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:54.798388 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:54.867100 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1422/2856] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:28:54.871540 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:54.932667 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704838-04da-4a28-b72c-b99c876e03bd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:54.937452 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:54.938052 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:54.938332 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:54.938605 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:54.939308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:54.941288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.056539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1435/2857] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:28:54 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:28:55.060794 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:55.071579 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b46dd-835c-4d6d-841d-01cf87e047a2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:55.140225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:55.140895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.141233 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.141563 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.142196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:55.144682 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.255798 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1423/2858] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:28:55 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:28:55.260062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:55.271612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d373e8-33b2-4247-86f5-34d9b679c9fb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:55.276441 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:55.277159 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.277628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.278343 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.278700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:55.280396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.295713 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1436/2859] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:28:55.297724 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:55.309864 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3b4ea-8972-4618-bbde-c11a27bb93c9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:55.315349 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:55.316114 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.316487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.316879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] PATH_INFO: `/v3/projects/8b5c228801df445e8f152241d673f352/users/1d29f76f312e4ea38d4f7eda44137dac/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.332789 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:55.336109 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5c228801df445e8f152241d673f352, user_id=1d29f76f312e4ea38d4f7eda44137dac, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:55.338848 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.432813 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1424/2860] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:28:55 2022] PUT /identity/v3/projects/8b5c228801df445e8f152241d673f352/users/1d29f76f312e4ea38d4f7eda44137dac/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:55.433212 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:55.444794 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4836a1-a315-4f08-abe6-c23826c7b0d2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:55.492534 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:28:55.513391 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ce3f8-51cc-420a-84b4-bb58e8617b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:55.514538 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ce3f8-51cc-420a-84b4-bb58e8617b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.516105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ce3f8-51cc-420a-84b4-bb58e8617b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.516105 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ce3f8-51cc-420a-84b4-bb58e8617b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.517153 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ce3f8-51cc-420a-84b4-bb58e8617b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:55.519662 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ce3f8-51cc-420a-84b4-bb58e8617b29 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.541930 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1437/2861] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:28:55.546330 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:55.558469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fb0bb-ac4c-4463-bc49-492a62ad0746 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:55.565283 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:55.570212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.570212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.570212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] PATH_INFO: `/v3/projects/8b5c228801df445e8f152241d673f352/users/1d29f76f312e4ea38d4f7eda44137dac/roles/be98a6e301f144d88c992f65bca561fb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.570212 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5c228801df445e8f152241d673f352, user_id=1d29f76f312e4ea38d4f7eda44137dac, role_id=be98a6e301f144d88c992f65bca561fb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:55.572605 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.590721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1425/2862] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:28:55 2022] PUT /identity/v3/projects/8b5c228801df445e8f152241d673f352/users/1d29f76f312e4ea38d4f7eda44137dac/roles/be98a6e301f144d88c992f65bca561fb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:28:55.597722 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.598011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.598320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.630935 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] MFA Rules not processed for user `1d29f76f312e4ea38d4f7eda44137dac`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:28:55.632325 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:55.754288 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1438/2863] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:28:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:28:55.764385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:55.766744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:55.837409 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c30def-139c-4667-bf3b-51393f30b52f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:55.842852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:55.844801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:55.844801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:55.844801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:55.912440 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:55.913907 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:55.923744 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1426/2864] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:57.866411 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:57.868070 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:28:57.880840 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26167cbe-bbac-49ad-bd49-f28f748d5027 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:57.885327 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:57.885939 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:57.886213 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:57.886476 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:57.942220 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:57.943690 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:57.952311 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1439/2865] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:28:59.331311 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:28:59.339785 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50fee-31c1-4834-939a-0819f1dd0843 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:28:59.343304 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8453b0-28ac-43e5-b882-6db7d63608eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:28:59.343887 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8453b0-28ac-43e5-b882-6db7d63608eb service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:28:59.344162 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8453b0-28ac-43e5-b882-6db7d63608eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:28:59.344427 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8453b0-28ac-43e5-b882-6db7d63608eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:28:59.353059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8453b0-28ac-43e5-b882-6db7d63608eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:28:59.354480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8453b0-28ac-43e5-b882-6db7d63608eb service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:28:59.363775 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1427/2866] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:28:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:03.900139 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:03.961792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617135-20dd-4cf2-b09e-f5961524a3e3 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:03.966397 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:03.967044 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:03.967321 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:03.967592 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] PATH_INFO: `/v3/users/b22ad986d925481e8e5fe661eea0a783` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:03.968704 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22ad986d925481e8e5fe661eea0a783)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:03.970608 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:03.975665 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:04.169745 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22ad986d925481e8e5fe661eea0a783'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:04.181839 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22ad986d925481e8e5fe661eea0a783'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:04.188537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22ad986d925481e8e5fe661eea0a783'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:04.189201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22ad986d925481e8e5fe661eea0a783'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:04.189806 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22ad986d925481e8e5fe661eea0a783'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:04.265876 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1440/2867] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:29:03 2022] DELETE /identity/v3/users/b22ad986d925481e8e5fe661eea0a783 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:04.270502 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:04.271066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:04.271066 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:04.295963 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:04.297738 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:04.358565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1428/2868] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:04.368970 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:04.370374 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:04.427400 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c787dd-258e-450b-9eeb-4bbdf7566746 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:04.432541 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:04.433252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:04.433627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:04.433896 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:04.443773 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:04.445253 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:04.455119 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1441/2869] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:04.844295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:04.845970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:04.858647 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2c6f8-3c58-4fef-9468-29108b57c4db None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:04.863385 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:04.864065 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:04.864344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:04.864613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] PATH_INFO: `/v3/projects/205e6728f1d04fcebaa70f337be35775` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:04.865809 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=205e6728f1d04fcebaa70f337be35775)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:04.867774 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:04.884238 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:04.974949 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205e6728f1d04fcebaa70f337be35775', 'request_id': 'req-d8e0130b-203f-4827-bed6-a9b7682b1387'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.037045 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205e6728f1d04fcebaa70f337be35775', 'request_id': 'req-d8e0130b-203f-4827-bed6-a9b7682b1387'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.043329 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205e6728f1d04fcebaa70f337be35775', 'request_id': 'req-d8e0130b-203f-4827-bed6-a9b7682b1387'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.054596 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] The token cache is being invalidate because project 205e6728f1d04fcebaa70f337be35775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:29:05.054903 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.055525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.056976 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1429/2870] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:29:04 2022] DELETE /identity/v3/projects/205e6728f1d04fcebaa70f337be35775 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:05.061106 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:05.071503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd146f-bf98-4d4e-950a-316e90bf9cda service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:05.126965 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:05.128088 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:05.128365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:05.128623 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] PATH_INFO: `/v3/users/5ad62d7441ba4942848cc34a235ff619` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:05.129850 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad62d7441ba4942848cc34a235ff619)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:05.131778 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:05.233120 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:05.348350 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad62d7441ba4942848cc34a235ff619'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.360154 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad62d7441ba4942848cc34a235ff619'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.367666 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad62d7441ba4942848cc34a235ff619'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.368370 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad62d7441ba4942848cc34a235ff619'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.371140 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad62d7441ba4942848cc34a235ff619'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:05.475445 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1442/2871] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:29:05 2022] DELETE /identity/v3/users/5ad62d7441ba4942848cc34a235ff619 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:05.928246 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:05.930600 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:05.944544 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0130b-203f-4827-bed6-a9b7682b1387 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:06.001944 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:06.002585 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:06.002895 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:06.003170 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] PATH_INFO: `/v3/projects/e9173d501b88458ca40dce2960901f18` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:06.004295 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9173d501b88458ca40dce2960901f18)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:06.006217 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:06.115810 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9173d501b88458ca40dce2960901f18', 'request_id': 'req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.179745 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9173d501b88458ca40dce2960901f18', 'request_id': 'req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.185508 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9173d501b88458ca40dce2960901f18', 'request_id': 'req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.196691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] The token cache is being invalidate because project e9173d501b88458ca40dce2960901f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:29:06.196912 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.197538 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.199253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1430/2872] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:29:05 2022] DELETE /identity/v3/projects/e9173d501b88458ca40dce2960901f18 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:06.203256 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:06.205210 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:06.218597 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b7405-2371-44f4-a750-702bd620884c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:06.273011 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:06.273642 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:06.273923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:06.274197 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] PATH_INFO: `/v3/users/1d29f76f312e4ea38d4f7eda44137dac` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:06.275330 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d29f76f312e4ea38d4f7eda44137dac)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:06.277362 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:06.460824 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d29f76f312e4ea38d4f7eda44137dac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.466668 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d29f76f312e4ea38d4f7eda44137dac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.474206 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d29f76f312e4ea38d4f7eda44137dac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.475025 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d29f76f312e4ea38d4f7eda44137dac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.475705 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e8fd6302-77f9-49cf-975f-51a1f68e6d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d29f76f312e4ea38d4f7eda44137dac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:06.550371 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1443/2873] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:29:06 2022] DELETE /identity/v3/users/1d29f76f312e4ea38d4f7eda44137dac => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:07.009174 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.010819 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:07.026622 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25ff68-e7f8-4ac6-8cfb-78490676d79e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.059923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:07.086968 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.087590 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.087868 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.088141 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] PATH_INFO: `/v3/projects/8b5c228801df445e8f152241d673f352` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.089258 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5c228801df445e8f152241d673f352)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.091176 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.228839 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5c228801df445e8f152241d673f352', 'request_id': 'req-ee450bf3-8968-4cf9-ae71-f8932d2826b2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:07.293704 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5c228801df445e8f152241d673f352', 'request_id': 'req-ee450bf3-8968-4cf9-ae71-f8932d2826b2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:07.300038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5c228801df445e8f152241d673f352', 'request_id': 'req-ee450bf3-8968-4cf9-ae71-f8932d2826b2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:07.311627 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] The token cache is being invalidate because project 8b5c228801df445e8f152241d673f352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:29:07.311914 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:07.312565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ee450bf3-8968-4cf9-ae71-f8932d2826b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:07.314030 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1431/2874] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:29:07 2022] DELETE /identity/v3/projects/8b5c228801df445e8f152241d673f352 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:07.319020 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.319536 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.319650 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.338792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:07.339977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:07.358888 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:07.399533 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1444/2875] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:07.409281 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.409564 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.409893 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.430084 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:07.431318 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:07.439577 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1432/2876] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:07.443947 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.454038 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a0475-5874-4a92-a623-941218e2b2b7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.459201 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.459792 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.460072 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.460346 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.461018 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:29:07.461320 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.462931 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.475308 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1445/2877] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 10 23:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:29:07.479642 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.490259 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e444-e91c-4446-88fc-81903e2a2f31 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.494885 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.495469 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.495740 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.496003 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.496614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.498484 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.592906 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1433/2878] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 10 23:29:07 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:29:07.597629 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.607510 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8494d71-5623-490c-b163-18c1edfe5217 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.665530 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.666134 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.666412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.666711 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.667446 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.669279 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.773027 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1446/2879] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 10 23:29:07 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:29:07.779034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.790792 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de426660-6bb7-4982-8969-553edb515e9f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.796122 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.796711 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.796984 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.797250 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.797879 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.799480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.809691 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1434/2880] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:29:07.813821 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.824036 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e69607-8134-4e27-94d7-07d530ee85b7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.828944 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59cde-c7b6-404f-a65d-469a13ce4774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.829560 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59cde-c7b6-404f-a65d-469a13ce4774 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.831003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59cde-c7b6-404f-a65d-469a13ce4774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.831003 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59cde-c7b6-404f-a65d-469a13ce4774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.832758 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f59cde-c7b6-404f-a65d-469a13ce4774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.835455 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f59cde-c7b6-404f-a65d-469a13ce4774 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.849914 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1447/2881] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 10 23:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:29:07.854431 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:07.864476 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd0b44-9945-43b4-a1af-57500a2ba36c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:07.869043 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:07.869679 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.869957 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.870228 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] PATH_INFO: `/v3/projects/76c66a7fe733409a88b4584a7e10bb4f/users/892815f6b41f4ec3acb52275ace9a172/roles/f8c74cd9c7d245249553a39801f08d82` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.884485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c66a7fe733409a88b4584a7e10bb4f, user_id=892815f6b41f4ec3acb52275ace9a172, role_id=f8c74cd9c7d245249553a39801f08d82)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:07.887155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:07.968299 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1435/2882] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 10 23:29:07 2022] PUT /identity/v3/projects/76c66a7fe733409a88b4584a7e10bb4f/users/892815f6b41f4ec3acb52275ace9a172/roles/f8c74cd9c7d245249553a39801f08d82 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:07.977743 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:07.978124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:07.978458 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:07.997588 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] MFA Rules not processed for user `e793eb6ffc5c41a282b74540f29c0a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:07.998891 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:08.055819 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1448/2883] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:08.066049 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:08.067858 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:08.123275 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc5deb-7056-44d2-b3e2-f99be221f2f4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:08.126855 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada3783-aa4f-4181-b95e-627cc7b34160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:08.127950 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada3783-aa4f-4181-b95e-627cc7b34160 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:08.128225 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada3783-aa4f-4181-b95e-627cc7b34160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:08.128487 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada3783-aa4f-4181-b95e-627cc7b34160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:08.137118 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada3783-aa4f-4181-b95e-627cc7b34160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:08.138525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada3783-aa4f-4181-b95e-627cc7b34160 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:08.148340 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1436/2884] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:09.326499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:09.341866 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb02a54-d006-4aff-b8ca-7a9921e59ebb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:09.347584 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:09.349700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:09.349700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:09.349700 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:09.353559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:09.365151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:09.367200 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:09.378915 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1449/2885] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:09.762265 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e3a5f-feb4-41ae-ba28-a79633b9a5a0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:09.762873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e3a5f-feb4-41ae-ba28-a79633b9a5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:09.762873 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e3a5f-feb4-41ae-ba28-a79633b9a5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:09.776131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:09.815143 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a30e3a5f-feb4-41ae-ba28-a79633b9a5a0 None None] MFA Rules not processed for user `892815f6b41f4ec3acb52275ace9a172`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:09.817931 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a30e3a5f-feb4-41ae-ba28-a79633b9a5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:09.934036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1437/2886] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:29:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:09.952166 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:09.954030 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:09.978309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:10.020907 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8bb9c-ac0c-4b99-851b-4fa7147f5f4d service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:10.024589 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b00d7-8b04-4a02-8f8a-9897c8ab3b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:10.025199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b00d7-8b04-4a02-8f8a-9897c8ab3b5d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:10.025474 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b00d7-8b04-4a02-8f8a-9897c8ab3b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:10.025740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b00d7-8b04-4a02-8f8a-9897c8ab3b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:10.035867 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b00d7-8b04-4a02-8f8a-9897c8ab3b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:10.037294 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b00d7-8b04-4a02-8f8a-9897c8ab3b5d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:10.046436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1450/2887] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:12.026456 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:12.027034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:12.027034 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:12.048062 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] MFA Rules not processed for user `892815f6b41f4ec3acb52275ace9a172`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:12.049301 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:12.056836 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1438/2888] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:29:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:20.108811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:20.109303 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:20.109410 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:20.124273 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:20.125733 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:20.130683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] MFA Rules not processed for user `bbf97a6003294b5ba411fa9ad2fda18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:29:20.131879 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:20.192126 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1451/2889] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 10 23:29:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:29:20.203083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390f0e-7540-4727-9b8c-1b10769caaa0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:20.210763 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:20.211539 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:20.211876 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:20.212210 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:20.214057 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:20.228853 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:20.230451 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:20.233138 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:20.234506 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:20.241970 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1439/2890] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:29:20.306244 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594c4f9-b2a0-4819-9a8a-3960a2017a0b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:20.311309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b58d64a6e04c1487f76e5ea26c74a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:20.311906 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:20.312184 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:20.312459 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:20.320596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:20.322169 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:20.330514 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1452/2891] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 10 23:29:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 23:29:25.014413 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:25.015956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:25.087790 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887e64a-aeb3-4e2c-89f7-e4b16958e472 service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:25.092852 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:25.093447 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:25.093731 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:25.093998 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:25.095807 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:25.107344 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:25.108923 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:25.114351 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:25.119069 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1440/2892] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:28.831959 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:28.845834 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d169-0e38-4398-80aa-bf522c7f0efa service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:28.852246 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:28.853087 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:28.853365 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:28.853635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:28.855735 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:28.868898 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:28.870559 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:28.882158 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1453/2893] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:29:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:29:28.973164 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:28.974721 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:28.989323 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25b8e-0f72-4ead-8e2e-8712e3e2776b service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:28.994026 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:28.994614 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:28.994929 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:28.995204 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] PATH_INFO: `/v3/users/ef47061e975849c6ab4054f380f8527c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:28.996319 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef47061e975849c6ab4054f380f8527c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:28.998294 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:29.209135 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef47061e975849c6ab4054f380f8527c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.221436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef47061e975849c6ab4054f380f8527c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.228953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef47061e975849c6ab4054f380f8527c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.229631 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef47061e975849c6ab4054f380f8527c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.230232 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef47061e975849c6ab4054f380f8527c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.332342 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1441/2894] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:29:28 2022] DELETE /identity/v3/users/ef47061e975849c6ab4054f380f8527c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:29.602057 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:29.614040 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e9b9-8150-47ea-8876-8ebcfab1a4e7 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:29.656612 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:29.672457 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:29.673075 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:29.678176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:29.678176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] PATH_INFO: `/v3/projects/1fbc9fef5e46490da9e1142d70e04228` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:29.678176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbc9fef5e46490da9e1142d70e04228)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:29.678176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:29.840416 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbc9fef5e46490da9e1142d70e04228', 'request_id': 'req-cc99c26c-6759-4526-ad9c-27b6a0e27696'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.989064 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbc9fef5e46490da9e1142d70e04228', 'request_id': 'req-cc99c26c-6759-4526-ad9c-27b6a0e27696'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:29.995453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbc9fef5e46490da9e1142d70e04228', 'request_id': 'req-cc99c26c-6759-4526-ad9c-27b6a0e27696'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.007686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] The token cache is being invalidate because project 1fbc9fef5e46490da9e1142d70e04228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:29:30.007975 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.010780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.010780 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1454/2895] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:29:29 2022] DELETE /identity/v3/projects/1fbc9fef5e46490da9e1142d70e04228 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:30.014368 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:30.015962 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:30.029603 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c28d65-09b4-40a4-ad88-3e247c465522 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:30.089305 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:30.089838 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:30.090111 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:30.090377 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] PATH_INFO: `/v3/users/fbfbc5309a5a4271a81f9a6ae0ffba1a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:30.091566 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbfbc5309a5a4271a81f9a6ae0ffba1a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:30.093485 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:30.300857 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbfbc5309a5a4271a81f9a6ae0ffba1a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.312793 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbfbc5309a5a4271a81f9a6ae0ffba1a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.320071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbfbc5309a5a4271a81f9a6ae0ffba1a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.320796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbfbc5309a5a4271a81f9a6ae0ffba1a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.321396 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ffefd501-d430-4db5-9f6a-597bf791ebd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbfbc5309a5a4271a81f9a6ae0ffba1a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.323552 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:30.414163 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1442/2896] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:29:30 2022] DELETE /identity/v3/users/fbfbc5309a5a4271a81f9a6ae0ffba1a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:30.672918 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:29:30.674436 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:29:30.689757 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99c26c-6759-4526-ad9c-27b6a0e27696 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:29:30.752130 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:29:30.752855 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:29:30.753176 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:29:30.753453 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] PATH_INFO: `/v3/projects/c0e869053ff848bfba779adcb43e6a37` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:29:30.754578 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e869053ff848bfba779adcb43e6a37)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:29:30.756498 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:29:30.877124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e869053ff848bfba779adcb43e6a37', 'request_id': 'req-cdfc60ef-caed-4da0-b004-d310b74945ce'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.962503 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e869053ff848bfba779adcb43e6a37', 'request_id': 'req-cdfc60ef-caed-4da0-b004-d310b74945ce'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.968805 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e869053ff848bfba779adcb43e6a37', 'request_id': 'req-cdfc60ef-caed-4da0-b004-d310b74945ce'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.980859 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] The token cache is being invalidate because project c0e869053ff848bfba779adcb43e6a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:29:30.981266 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.981917 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:29:30.983482 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1455/2897] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:29:30 2022] DELETE /identity/v3/projects/c0e869053ff848bfba779adcb43e6a37 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:29:40.491708 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:29:40.977596 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:30:36.514418 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:30:36.515038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:30:36.515038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:30:36.534317 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] MFA Rules not processed for user `892815f6b41f4ec3acb52275ace9a172`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:30:36.535550 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:36.588442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1443/2898] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:30:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:30:36.829360 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:30:36.830820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:36.885887 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc60ef-caed-4da0-b004-d310b74945ce admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:30:36.889968 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:30:36.890574 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:30:36.890928 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:30:36.891199 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:30:36.892881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:36.902470 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:30:36.903943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:30:36.911943 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1456/2899] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:30:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:30:37.310184 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:30:37.312276 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:37.390306 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9535a96-68b7-4740-b517-4ecd0cb23291 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:30:37.395270 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:30:37.395865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:30:37.396146 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:30:37.396424 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:30:37.398139 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:37.409892 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:30:37.411507 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:30:37.422035 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1444/2900] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:30:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:30:46.908341 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=26 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:30:47.417108 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=57 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:30:48.008644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:30:48.010435 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:48.084769 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62e592-9ecd-48bd-bba9-27eb29687e28 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:30:48.090216 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:30:48.090846 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:30:48.091124 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:30:48.091393 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:30:48.093090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:30:48.108150 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:30:48.109811 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a538937-c47e-403e-8403-15d1ec4de657 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:30:48.122627 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1457/2901] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:30:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:30:58.116820 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=34 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:31:11.884152 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:31:11.885612 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:31:11.945442 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09308-e572-4ff8-ad70-d599cd651f22 service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:31:11.949391 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54fff64dfcf8434d867c028a72c17a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:31:11.949981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:31:11.950253 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:31:11.950806 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:31:11.952700 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:31:12.015958 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:31:12.017509 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:31:12.030759 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1445/2902] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:31:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:31:22.022429 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=48 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:31:25.668132 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:31:25.668648 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:31:25.668808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:31:25.687889 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] MFA Rules not processed for user `05e18a4fd73e435aa6d0f10a37405698`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:31:25.689023 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:31:25.695957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1458/2903] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 10 23:31:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:31:26.008557 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:31:26.018316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941cd46-b6c4-4564-b975-9e716707fe3d service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:31:26.021856 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:31:26.022435 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:31:26.022739 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:31:26.023038 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:31:26.024702 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:31:26.034597 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:31:26.036037 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:31:26.043933 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1446/2904] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:31:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:31:35.682615 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:31:36.040362 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:31:50.476923 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:31:50.497675 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aa191-f79b-4de7-b9cf-e0d422a2b64e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:31:50.501433 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:31:50.502022 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:31:50.503009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:31:50.503009 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:31:50.504381 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:31:50.600490 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:31:50.602089 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:31:50.612415 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1459/2905] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:31:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:31:52.579131 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:31:52.621036 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc1b53-6113-4433-9975-0d77885b397f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:31:52.626384 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:31:52.627018 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:31:52.627296 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:31:52.627565 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:31:52.629346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:31:52.658801 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:31:52.660403 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:31:52.671881 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1447/2906] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:31:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:32:00.387001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:32:00.393905 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:32:00.454635 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac442-07df-4fe5-80d5-278ed4d62ec2 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:32:00.459499 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:32:00.460131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:32:00.460412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:32:00.460683 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] PATH_INFO: `/v3/users/892815f6b41f4ec3acb52275ace9a172` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:32:00.461826 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=892815f6b41f4ec3acb52275ace9a172)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:32:00.463853 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:32:00.760860 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892815f6b41f4ec3acb52275ace9a172'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:00.768024 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:32:00.773151 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892815f6b41f4ec3acb52275ace9a172'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:00.780628 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892815f6b41f4ec3acb52275ace9a172'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:00.781323 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892815f6b41f4ec3acb52275ace9a172'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:00.781919 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892815f6b41f4ec3acb52275ace9a172'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:00.871006 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1460/2907] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:32:00 2022] DELETE /identity/v3/users/892815f6b41f4ec3acb52275ace9a172 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:32:01.131353 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:32:01.142862 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7e91-7e18-456a-bf32-6fe7a30aee33 service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:32:01.202953 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:32:01.203578 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:32:01.203854 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:32:01.204126 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] PATH_INFO: `/v3/projects/76c66a7fe733409a88b4584a7e10bb4f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:32:01.205240 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c66a7fe733409a88b4584a7e10bb4f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:32:01.207243 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:32:01.318053 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c66a7fe733409a88b4584a7e10bb4f', 'request_id': 'req-bdec32b9-2971-49ec-b307-a42362924839'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:01.415316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c66a7fe733409a88b4584a7e10bb4f', 'request_id': 'req-bdec32b9-2971-49ec-b307-a42362924839'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:01.425048 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c66a7fe733409a88b4584a7e10bb4f', 'request_id': 'req-bdec32b9-2971-49ec-b307-a42362924839'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:01.442546 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] The token cache is being invalidate because project 76c66a7fe733409a88b4584a7e10bb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:32:01.443014 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:01.443935 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:32:01.446059 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1448/2908] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:32:01 2022] DELETE /identity/v3/projects/76c66a7fe733409a88b4584a7e10bb4f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:32:10.900881 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=9,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:32:11.437486 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:33:53.753173 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:33:53.755131 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:33:53.876908 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde6dd9-bfc2-49b9-8e35-6edf7b4c2cad admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:33:53.944689 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cef180b05347ebbcc6f3a6e97dac3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:33:53.945300 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:33:53.945575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:33:53.945845 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:33:53.947872 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:33:53.960396 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:33:53.961977 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:33:53.972799 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1461/2909] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:33:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:33:54.203619 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:33:54.205436 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:33:54.284548 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec32b9-2971-49ec-b307-a42362924839 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:33:54.289382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:33:54.289981 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:33:54.290261 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:33:54.290518 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:33:54.292346 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:33:54.306369 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:33:54.307865 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:33:54.319551 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1449/2910] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:33:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:33:55.528641 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:33:55.538575 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7a4e-4158-4afb-a3a3-a3e8bdc590b5 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:33:55.542260 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:33:55.542874 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:33:55.543146 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:33:55.543412 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:33:55.545137 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:33:55.558282 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:33:55.559775 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:33:55.571392 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1462/2911] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:33:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:34:04.314916 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=35 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:34:05.565691 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=80 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:34:06.035024 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:34:06.043888 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dc505-8166-47e2-b247-2fc924a22093 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:34:06.047495 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4337750a9fb940a387400d1f0ca9ba02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:34:06.048076 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:34:06.048348 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:34:06.048613 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:34:06.050293 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:34:06.133611 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:34:06.135083 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:34:06.153382 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1450/2912] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:34:13.266466 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:34:13.268405 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:34:13.282252 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2045e44-0852-4b17-80c3-cf093d3ff4c5 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:34:13.287447 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:34:13.288071 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:34:13.288351 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:34:13.288625 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] PATH_INFO: `/v3/users/05e18a4fd73e435aa6d0f10a37405698` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:34:13.289746 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e18a4fd73e435aa6d0f10a37405698)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:34:13.291686 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:34:13.471049 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e18a4fd73e435aa6d0f10a37405698'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.483475 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e18a4fd73e435aa6d0f10a37405698'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.490304 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e18a4fd73e435aa6d0f10a37405698'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.491008 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e18a4fd73e435aa6d0f10a37405698'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.491644 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e18a4fd73e435aa6d0f10a37405698'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.547967 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1463/2913] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:34:13 2022] DELETE /identity/v3/users/05e18a4fd73e435aa6d0f10a37405698 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:34:13.767327 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:34:13.779104 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c61a8-dac1-49b4-be0c-f684a9bcb289 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:34:13.836271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:34:13.836884 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:34:13.837155 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:34:13.837420 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] PATH_INFO: `/v3/projects/8d2bfa865669477f8757e977da29420a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:34:13.838510 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2bfa865669477f8757e977da29420a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:34:13.840415 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:34:13.936545 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2bfa865669477f8757e977da29420a', 'request_id': 'req-5f876730-e956-41a7-b972-0a0e97e58f8e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.993788 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2bfa865669477f8757e977da29420a', 'request_id': 'req-5f876730-e956-41a7-b972-0a0e97e58f8e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:13.999637 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2bfa865669477f8757e977da29420a', 'request_id': 'req-5f876730-e956-41a7-b972-0a0e97e58f8e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.001471 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:34:14.011428 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] The token cache is being invalidate because project 8d2bfa865669477f8757e977da29420a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:34:14.011641 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.012271 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.013799 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1451/2914] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:34:13 2022] DELETE /identity/v3/projects/8d2bfa865669477f8757e977da29420a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:34:14.017808 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:34:14.019309 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:34:14.032953 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c482a55-3f78-460d-9d4a-c951a3cf3bf4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:34:14.089539 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:34:14.090165 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:34:14.090443 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:34:14.090740 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] PATH_INFO: `/v3/users/6d8cf8de7c644af78e059460ff804840` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:34:14.091856 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8cf8de7c644af78e059460ff804840)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:34:14.093759 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:34:14.282316 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8cf8de7c644af78e059460ff804840'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.294978 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8cf8de7c644af78e059460ff804840'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.301957 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8cf8de7c644af78e059460ff804840'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.307090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8cf8de7c644af78e059460ff804840'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.307090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8cf8de7c644af78e059460ff804840'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:14.307090 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:34:14.388785 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1464/2915] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 10 23:34:14 2022] DELETE /identity/v3/users/6d8cf8de7c644af78e059460ff804840 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:34:14.641602 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:34:14.643071 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:34:14.657316 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f876730-e956-41a7-b972-0a0e97e58f8e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:34:14.724796 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e793eb6ffc5c41a282b74540f29c0a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '669ea2e08a3a4675928c19c0f5a75735', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:34:14.725411 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:34:14.725686 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:34:14.725956 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] PATH_INFO: `/v3/projects/bfbaff8da48c497e8c8d04905de027be` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:34:14.727090 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbaff8da48c497e8c8d04905de027be)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:34:14.728973 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:34:14.984455 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbaff8da48c497e8c8d04905de027be', 'request_id': 'req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:15.125525 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbaff8da48c497e8c8d04905de027be', 'request_id': 'req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:15.132129 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbaff8da48c497e8c8d04905de027be', 'request_id': 'req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:15.143196 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] The token cache is being invalidate because project bfbaff8da48c497e8c8d04905de027be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:34:15.143480 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:15.144198 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e2549a6-a8cc-4856-aa48-b0d28fff0b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:34:15.145687 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1452/2916] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 10 23:34:14 2022] DELETE /identity/v3/projects/bfbaff8da48c497e8c8d04905de027be => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:34:24.582854 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:34:25.140117 dv-cinder-851449-5-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:36:54.524062 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:36:54.525573 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:36:54.594673 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7084-778f-4642-a0a7-49519ec7e472 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:36:54.654537 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604f58c168e432d812608a2351610bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7defdf2789ed4b3b85bc47aa4c32f317', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:36:54.655243 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:36:54.655520 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:36:54.655786 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:36:54.657485 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:36:54.669037 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:36:54.670587 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56809d8-40b7-4cb1-8b9d-ab13f5994e92 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:36:54.680230 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1465/2917] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 10 23:36:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:37:04.676001 dv-cinder-851449-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=58 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}